Supporting Consensus-based Software Development: a Vision Paper

Abstract

-Traditional, vertical organizational models of software development have been challenged by more agile and collaborative structures. Recently, this has also been demonstrated in the emergence of explicitly horizontalist organizational structures, focused on consensus-based decision making. In this paper, we describe the principles and processes of these “ConsensusBased Communities” (CBCs) and outline the main challenges they face as they try to implement “Consensus-Based Software Development” (CBSD). We express these as early, high level requirements for a tool supported methodology. Based on these, we present and analysis of existing tools that shows that no single tool provides complete support for consensus-based group decision making. We thus outline directions for future research, identifying opportunities for the development and deployement of model-based techniques in this emergent field.

I. INTRODUCTION

Software development is a team activity [ 1 ], [ 2 ], in which internal team dynamics impact the quality of the produced software. A classic formulation of this is “Conway’s Law”: Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure. [ 3 ] More recently, Nagappan et al. showed that organizational structure was a better predictor of fault-proneness than traditional technical metrics [ 4 ]. A defining aspect of a team’s organization and communication structure is Group Decision Making (GDM), also known as Multi-Person Decision-Making (MPDM), [ 5 ], [ 6 ], [ 7 ]. Traditionally, organization structures are hierarchical and command based. However, the advent of Agile software development has changed GDM processes, as it tends to prefer horizontal structures [ 8 ]. Characteristically, the Agile Manifesto declares:

The best architectures, requirements, and designs emerge from self-organizing teams. [ 9 ]

In practice, the degree of self-organization varies widely across different organizations. We illustrate this diversity in Figure 1, as a continuum of organizational horizontality in software development teams. On the one end there is traditional “command-based” development, where upper levels of the organizational hierarchy have the final say on what developers do. Moving from that extreme, some organizations maintain the use of project managers who have the final say in decisions while encouraging the building of a team consensus. Some organizations actively pursue a “flat” model [ 10 ] or an “holarchy” [ 11 ], where decisions are delegated to selfmanaging teams. Often, organizations using self-managed teams still maintain a higher authority, simply reducing the amount of middle management. In Figure 1 we position Agile teams, that promote self-organized GDM, in the middle of the continuum, along with flat and holarchic organizations.

In this paper, we put the spotlight on the other end of the continuum. This is populated by software development organizations that intentionally adopt explicitly anti-hierarchical and consensus-based modes of GDM and collaboration. For example, Koumbit, a web-hosting company based in Montreal, describes its internal processes as “non-hierarchical selfmanagement” [ 12 ]. Loomio is a developer-owned cooperative based in New Zealand that puts consent at the centre of its decision making process [ 13 ]. Other organizations like Zappos [ 14 ] and Medium.com [ 15 ] maintain horizontal principles. This is also often the case for open source communities, such as Drupal, which favours a self-managed approach [ 16 ]. We call such organizations Consensus-Based Communities (CBCs).

CBCs face a set of unique challenges when developing software, mostly based on their philosophical approach to GDM. The Agile Manifesto already pushed GDM away from the “Benevolent Dictator” management paradigm.

Agile Software Development has completely dispensed with the formal job title of the project manager. Agile methodologies such as scrum seem to distribute the erstwhile responsibilities of the project manager into new roles. [ 8 ] CBCs affirm that team decisions must go a step further, and ensure consensus building as much as possible, instead of majority decision-making. The rationale behind striving for consensus is based on a perceived flaw of democratic decisionmaking called “tyranny of the majority”:

The will of the majority may be seen as the will of the whole group, with the minority expected to accept and carry out the decision, even if it is against their deeply held convictions and most basic needs. It is possible for a voting group to look for solutions that would suit everybody, but it is more common for ideas with a majority backing to be pushed through. [17, p. 9] This flaw can cause, for example a team with a majority of non-experts to overrule a minority of experts, leading to poor decisions. Instead, consensus-building requires team members to convince all their colleagues, instead of unilaterally imposing their propositions. This means that software developers in CBCs cannot exploit the process in order to shut down their colleagues, but must provide facts and data able to persuade them to change their mind.

CBCs such as the ones mentioned above also operate under anti-hierarchical principles. Their perception is that software development has become so complex that not one person can see the whole picture. Each software developer is therefore a piece of the puzzle, each with their own expertise, and decisions should then be made with everyone in mind [ 7 ].

Even though CBCs are currently not the norm in software engineering, studying them can provide interesting insights on team dynamics, and can help identify beneficial organizational practices across the continuum of organizational horizontality. Broadly speaking, the recent move away from traditional organizational practices can be understood as a concern about the degree to which they are appropriate for software development. Thus, interesting insights can be obtained from radically different organizational structures [ 10 ]. At the same time, since CBCs are software producing organizations in an era of software ecosystems [ 18 ], understanding the impact of their practices on software quality (cf. Conway’s Law) may have ramifications beyond their niche.

In this paper, we focus on CBCs and propose a vision for a comprehensive, model-based, tool supported methodology for Consensus-Based Software Development (CBSD) that addresses their unique set of challenges. We identify the specific needs of CBCs and study existing collaboration tools to find gaps between the two and potential avenues of research and development. Specifically, we make the following contributions: a) We highlight the existence of CBCs and describe their unique processes and challenges; b) We describe how consensus-based GDM can be supported for software development; c) We analyze the capabilities of existing tools, identify their limitations, and outline how model-based techniques can be used to improve the state of CBSD practise.

The rest of the paper is organized as follows: We present background information on CBCs in Section II, detailing their typical internal processes In Section III, we explain the challenges raised by consensus-based GDM and in Section IV we outline the requirements for a tool framework for supporting consensus-based software development. We analyze the capabilities of existing GDM support tools in Section V and point out areas for improvement in Section VI. We discuss related work in Section VII and conclude in Section VIII.

II. BACKGROUND

To better understand the unique challenges in CBSD, we first outline the main characteristics of CBCs.

Consensus-based decision-making does not mean that everybody must agree on everything. In practice, consensus is typically “soft”, which means that certain forms of disagreement are acceptable. According to the Consensus Handbook [17, p. 6]:

Instead of simply voting for an item and having the majority of the group get their way, a consensus group is committed to finding solutions that everyone actively supports, or at least can live with.

The aim is to avoid a situation where there are winners and losers. CBCs claim that this helps everybody feel involved in the organization, and helps them learn new skills and be better, more complete software developers [ 12 ]. This further increases the sense of agency and ownership towards decisions, since each individual actively participates, ensuring that everyone’s concerns are part of the process. A major benefit of consensus decision-making is therefore the increased solidarity among team members in cases where bad decisions are made. Faced with a negative result, since the whole team was engaged in the decision together, no single individual or group can be singled out for blame [ 19 ].

In the following, we describe a typical consensus-based GDM process, in terms of the activities performed and the roles and responsibilities associated. Consensus-based decision-making processes are adapted from the different kinds of democratic practices surrounding them [ 20 ]. They are adapted to the size and ability of the group.

While the exact process varies from one organization to another, we present here a basic model of typical activities in consensus-based GDM [17, p. 16]. These are: Introduction (A1): Introduction of the issue on which a decision must be made; Discussion (A2): Discuss the issue and collect ideas; Concerns (A3): List concerns regarding the proposal; Proposal (A4): Identify emerging proposals meeting concerns; Refinement (A5): Discuss, clarify and amend the proposals; Consensus test (A6): Test for agreement, usually through a straw poll. If not, return to the previous points; Implementation (A7): Implement the decision.

We show these activities diagrammatically in Figure 2.

During a test for consensus (activity A6), a typical CBC uses four types of votes for the straw poll [17, p. 16] with the following meaning: Block (V1): The participant thinks the proposal is fundamentally wrong and should not be implemented by the organization.

Stand aside (V2): The participant cannot support the proposal and is not willing to implement it. However they V1. Block are willing to let the organization implement it without them.

Reservation (V3): The participant has some reservations, but is willing to let the proposal be implemented and to work for its implementation.

Agreement (V4): The participant supports the proposal and is willing to implement it.

The responsibilities associated with the good functioning of the team are traditionally assigned to a team manager. Instead, in self-managing teams, they must be delegated to team members. Consensus-based teams recommend to split responsibilities to different people to avoid a concentration of powers which would re-create a hierarchical structure [ 21 ]. Specific roles include [ 17 ], [ 21 ]: Facilitators (R1): A neutral party whose responsibility is to keep the meeting on topic and assist the decision-making process. In some cases, no single person is assigned the role, instead relying on the participants, assuming that they have internalized it enough to self-discipline. Minute Takers (R2): Responsible for keeping track of the decisions and action items.

Hand Takers (R3): Responsible of keeping track of whose turn it is to speak next and control the time allotted to each speaker.

Participant (R4): Each team member is responsible for selfdiscipline and for the good functioning of the process.

Facilitation is a complex role, which can be broken down into the following responsibilities [17, p. 31]: Active listening (F1): To assist speakers in expressing their ideas and its associated context.

Summarising (F2): To reformulate the intervention of the speaker in a clear and concise manner in order to confirm its interpretation.

Synthesis (F3): To merge the ideas in actual proposals potentially acceptable for all participants.

The list of work products of a consensus-based process includes the minutes of the meeting, along with the decisions and action items. It is recommended that action items be tracked independently from the rest in order to avoid losing track of them and to prevent power concentration into the same hands [ 21 ].

III. CHALLENGES OF CONSENSUS-BASED GDM Software development teams that adopt a consensus-based GDM process face a unique set of challenges. These stem from the need to work and reach consensus. Degenerations of the process can introduce problems such as false consensus, where an explicit agreement is reached, but where team members implicitly disagree. We group such challenges along seven different dimensions.

One dimension (D1) is related to the heterogeneity of communication [ 7 ]. For example, software engineering teams include a variety of experts of different fields, depending on the needs of the project. These different experts can be proficient in different languages and techniques. For example, one expert can be well-versed with the UML language while the rest of the team is not. This creates a hurdle for communication as the expert struggles to explain his point to non-experts.

Another dimension (D2) is related to the need to persuade others in order to obtain consensus. Ideally, this persuasion is based on real facts, brought by real experts, but in practice, emotional factors can enter the equation. For example, “people are easily persuaded by other people that they like” ([ 22 ], quoted by [ 23 ]). Another example, “consistently slandering someone behind his back” [21, p. 33] is another tactic to persuade others using unsavory means.

This leads to the third dimension (D3) of how to adequatly generate alternatives for a complex issue. Ideally, GDM follows a rational decision-making approach. This approach is akin to a breadth-first approach, where all alternatives are studied and compared. The opposite is the naturalistic decision-making approach. This approach is akin of a depthfirst approach, where one easy to find solution is studied indepth [ 24 ]. For example, a typical software development team mixes the two approaches, going in-depth with a small number of alternatives [ 25 ]. Generally this is because the complexity of software development context makes a breadth-first approach impossible and requires limiting the breadth to what seems possible at the time [ 24 ], [ 25 ].

Generating and choosing the proper alternatives can be difficult, especially with dysfunctional teams (D4). Issues of “groupthink”, where there is a reluctance of criticizing ideas, are well documented [ 26 ]. Instances of polarized factions constantly blocking the propositions of the others are also a symptom of a dysfunctional team [ 26 ], [ 27 ]. Even before such problems arise, social stratification due to factors such as racism, sexism and class, generally makes marginalized people take less share in the decision-making process. Worse, that phenomenon is often justified by those who are silencing themselves as they perceives themselves as less competent [ 28 ]. Creating a context for genuine participation is an ongoing process that requires patience and dedication.

D2.

Evaluate expertise and

D1.

Manage the difference

of languages

UC6.

Support whiteboarding

UC5.

Monitor decisions

UC1.

Present the issue

UC2.

Manage the proposals

UC3.

Document the arguments and and

F3.

Synthesize interventions

UC7.

Identify expertise

helps helps s p l e h helps

and helps Supporting the CBSD process helps helps

V1.

Record block votes

UC4.

Detect consensus and and

A6.

Test for consensus and and

V2.

Record stand aside votes

V3.

Record reservation votes and

A7. and Implement the decision

D5. and Self-monitor decisions

V4.

Record agreement votes

A1.

Introduce the issue

and

A2.

Discuss the issue and

and and and

and

R2.

Take the minutes

and

R3.

Manage speakers

The fifth dimension is related to the nature of antihierarchical organizations (D5). Without a manager, the developers themselves must assume responsibility of the task typically assigned to managers. These include managing conflicting priorities between stakeholders, assuring that action items are actually implemented, and giving more weight to the intervention of experts [ 6 ], [ 10 ].

Another challenge is related to documenting the rationale of the decisions made (D6). As [ 29 ] writes:

In the field of software architecture, there has been a paradigm shift from describing the outcome of architecting process mostly described by component and connector (know-what) to documenting architectural design decisions and their rationale (know-how) which leads to the production of an architecture. [ 29 ] A GDM support tool should therefore document decision rationale to ensure that future developers can understand what arguments supported the decision. Ideally, such a tool would be able to see how decisions evolve [ 30 ]. For example, developers should know that a decision can be changed if the underlaying arguments are no longer valid.

But the technical context of the issue is not the only context to manage. Since CBCs are not the norm, the GDM process is generally a two-way educational process, first teaching newcomers about the GDM process, but also teaching ways to address concerns in a constructive manner. With consensusbased GDM, communities transition from a discursive space in which people can be either right or wrong to one where people are either more or less aware of the issues surrounding its practices. To make things more complicated, as is the case for any organization, the typical process is not followed to the letter, but changes are actually frequent: Since there are no leader to impose a process, any meeting can result in a process change. Hence, the bulk of the decision-making process is centred around educating participants on the current organizational context. Documentation of such processes is contextual and reflects a community’s ongoing organizational dynamics. Therefore, the given organizational context of the CBC (D7) will have an impact on its GDM, more so than in a traditional contexts using common GDM practices. The rationale of the decision (D6) cannot be understood outside this constantly evolving organizational context of the CBC (D7).

IV. REQUIREMENTS FOR CBSD SUPPORT TOOLS We identify the high-level requirements for CBSD tools, based on the processes, activities, roles, and artifacts involved in CBSD described in Section II, as well as the challenges outlined in Section III. We present the requirements in the form of use-cases, listed below. We also give the requirements as goal models in Figures 3 and 4 using i*, a language for modelling early requirements [ 31 ], [ 32 ].

UC1: During the introduction of an issue, present its context in a concise yet understandable way (A1).

UC2: Collect the ideas brought during the meeting (A2), obtain concerns from the participants and keep track of them (A3). Turn ideas into actionable proposals that everyone can inspect and amend (A4). Record amendments to proposals (A5).

UC3: Document the arguments pertaining to each proposal (A5) and the identity of the person(s) who brought the argument forward (R3). While it would be difficult to support active listening (F1) and the reformulation of Supporting

CBSD introspection helps

helps helps

UC9.

Detect dysfunctions and

D4.

Identify groupthink and other dysfunctional decision processes

UC10.

Understand decisions

D7.

Educate the organizational context and and

D6.

Document rationale Fig. 4: Use-cases to support for CBSD introspection, using the i* modeling language.

participants’ intervention (F2), the tool should assist the Facilitator’s synthesis responsibility (F3). Each decision should include the minutes of the relevant meetings in order to understand better the context of the decision (R2).

UC4: Detect consensus, through mechanisms like straw polls (A6). Straw polls should support the four types of votes (V1-V4). It should be possible to record the rationale of non-consenting votes (V1-V3).

UC5: Monitor the implementation of decisions after the meeting, to ensure it is carried out as decided (A7, D5). UC6: Record relevant sketches (e.g., whiteboard photos).

Discussion can often lead to a refined model that can be understood by all participants (D1).

UC7: Identify and evaluate the expertise relevant to the decision at hand (D2).

Introspection, reflection and self-improvement is a big part of the Agile philosophy [ 9 ], and the philosophy of CBCs. A CBSD support tool should also therefore support the following use cases: UC8: After implementing a decision, determine if the alternatives studied were adequate or if the team should have spent more time finding alternatives (D3).

UC9: Detect degenerated group decision making problems, such as groupthink, unresolved conflict, etc. (D4). UC10: Some decisions can have a long-term impact. Developers years later might have to deal with the decisions made today and might want to understand why the specific decision was made (D6). Understanding the decision means understanding the organizational context wherein the decision was made at the time. Participants must therefore be educated on the organizational context in order to understand the related decisions (D7).

V. WHAT IS CURRENTLY MISSING IN CBSD SUPPORT

TOOLS?

In this section we describe how existing tools and technologies fit with the CBSD activities, roles and work products described in previous sections. Ideally, CBSD support tools should be compatible with the processes presented earlier, and address the challenges of consensus-based decision-making. In practice, while some are covered by existing tools, other functionalities are still missing. The list of tools analyzed is a preliminary, non-exhaustive list of tools, based on a similar list published in the literature [ 33 ] as well as our own investigation.

ArguNet [ 34 ] and Carneades [ 35 ] are tools for structuring debates into arguments and logical propositions. They can record the ideas and arguments presented during a GDM meeting, using formal logic to encode their structure. They treat arguments as logically true propositions which can either support or oppose a proposal. They do not have voting mechanisms or support for evaluating expertise. ArguNet uses a directed node graph similar to concept models to link arguments and logicial propositions. Carneades, on the other hand, uses a tree modeling langage similar to i* models in order to structure arguments.

bCisive [ 36 ] is a tool for structuring debates. It provides a richer modelling language (over fifty pictograms) than ArguNet and Carneades. For example, it allows typing the sources of arguments, e.g., to differentiate between expert opinion and hearsay. Like Carneades, bCisive uses a tree modeling langage similar to i* models.

DebateGraph [ 37 ] is a mind-mapping tool offering the capability to structure data. It can link arguments and proposals, but its utility remains limited for GDM. Like ArguNet, it uses a directed node graph similar to concept models to link documents together.

Loomio [ 38 ] is a GDM tool useful for voting when there are already clearly-defined alternatives. It does not help in finding and structuring the debate in order to find these alternatives. It does not directly use a modeling language, but it uses simple graphs (pie charts, bar charts) to display the level of consensus.

Planning Poker [ 39 ] is useful for choosing an appropriate priority or estimate. Alternatives are clearly stated in the form of numerical scales. The facilitator can choose among various scales (such as the Fibonacci sequence, powers of 2, SmallMedium-Large, etc.) according to what is most appropriate for a given context. Planning poker uses a pictogrammic modeling langage based on common playing cards in order to facilitate understanding.

Reddit [ 40 ] and Discourse [ 41 ] are discussion tools, where participants bring arguments based on an introduction statement. Reddit integrates a voting mechanism which ensures that entries supported by more participants are ranked first. Discourse is closer to a mailing list and presents arguments in a temporal sequence. While Reddit does not use a modelling language, Discourse uses a simple pictogrammic model based on users’ avatars in order to show the state of a discussion. A Discourse discussion is modeled with the avatar of the initiator, and avatars of the last users to reply to the discussion.

holaSpirit [ 42 ] is an implementation of the “holacracy” [ 11 ] framework, which focuses on managing responsibilities and meeting agendas. It includes a way to document tensions between participants, which can help diagnose some dysfunctionalities. It can help identify expertise, based on the assignment of responsibilities. The “holacracy” framework relies on Venn diagrams to show how responsibilities can be regrouped into roles.

We summarize the results of our analysis in Table I. Note that the activity “Refinement” (A5) appears in two columns and does not present the same results. The reason is that one analysis is taken from the point of view of management proposals (UC2) while the other is based on argument documentation (UC3). For example, a tool can permit edition of proposals (A5 from UC2), but does not cover argument documentationm (A5 from UC3), therefore resulting in different results.

Studying the results presented in Table I, it is possible to pinpoint incompatibilities between current tool functionalities and the needs of CBSD. Some requirements are poorly covered by current tools, while others are not covered at all.

Results also show that many tools focus on one aspect of GDM exclusively. This is coherent with the current tendency away from a single tool able to do everything (e.g., Rational Rose) and toward multiple inter-communicating tools (e.g., integrating Slack with JIRA and Bitbucket). It would therefore theoretically be possible to cover the needs of CBSD using multiple inter-communicating tools. Unfortunately, most of these tools cannot do this right now, forcing a manual transfer of data from one tool to the next. For example, while none of the tools analyzed provides direct support for decision monitoring (UC5), it would be possible to support it through a connection to a project management tool like Microsoft Project or Atlassian JIRA.

However, one obstacle to this integration is the two widely different paradigm used by the tools analyzed. About half the tools studied support a rational decision process (e.g. ArguNet, bCisive, Carneades), rooted in logical propositions akin to Bayesian logic. These tools assume that arguments brought forward are equally true and build the GDM around argument structures. The other half support a naturalistic decision process (e.g. Loomio, Reddit, Planning Poker) and are rooted in fuzzy propositions. These fuzzy propositions are not assumed to be true, and GDM is therefore built around participants’ votes.

The importance of both rational and naturalistic approaches to GDM has been detailed in published literature [ 24 ], [ 25 ]. On the one hand, purely rational tools are limited in practice. On the other hand, while some fuzzy argumentation models exist, they focus on artificial intelligence and are not supported by any GDM tools [ 43 ]. This implies that a fuzzy argumentation modelling support tool might be useful for practical GDM, and in extension, to CBSD.

Results show only partial coverage of the “Refinement” activity (A5) from the point of view of proposal management (UC2). A few tools support the editing of proposals and arguments, but none keep track of changes. This requirement is important in practice because it is often necessary to come back to previous versions of proposals, e.g., when a change is blocked by participants. It is also important for postmortem evaluation of the GDM process, especially postmortem alternative evaluation (UC8) and team dysfunction identification (UC9).

Among the uncovered requirements, none of the analyzed tools support postmortem alternative evaluation (UC8). This is cause for concern since poorly framed problems are the source of multiple software catastrophes [ 44 ]. It would be interesting to track changes made to the alternatives throughout the project in order to detect when poor alternatives were chosen, or where radical changes to alternatives occurred during development.

No tools support the postmortem evaluation of team dysfunctions (UC9), except holaSpirit, which does it partially through the manual identification of tensions. With a tool able to keep track of changes, it would be possible to see how the decision evolved throughout the project. It would therefore be possible to perform some reasoning not only on one static model, but on a serie of evolving models. It seems however that at this time, no framework exists to study the evolution of argument models.

Given the importance given to the management of expertise in the literature [ 7 ], [ 6 ], [ 44 ], it is surprising that more tools are not taking it into account. Only two tools support it, but they do not implement voting mechanisms. In other words, expertise is not factored in any of the vote-based tools. This limits the capability of stakeholders to assess the relative merits of alternatives, as all proposals, arguments and votes are considered equal, which might not be the case in practice.

VI. GOING FORWARD

Given the mismatch between the capabilities of existing tools and the need of CBCs, we propose a roadmap for research on model-based techniques for supporting CBSD. We focus on model-based techniques specifically because the Model Driven Engineering (MDE) approach allows describing the various relevant artifacts at a high level of abstraction, which in turn helps integrate and analyze information from heterogeneous sources, such as discussions, decisions, design artifacts, documentation, source code, etc.

A. Main research questions

Research on CBSD should at a minimum address the following:

a) RQ1. Inter-tool communication:: Developers should be able to seamlessly use chains of relevant tools. For example, they could pick one tool for problem statement, a second for proposals and a third for argument documentation (UC1, UC2, UC3), and be able to link the decision data to a project management tool (UC5) of their choice. It is a classic problem in software engineering and model-driven development. We propose to use metamodelling and language globalization [ 45 ] to create an infrastructure for interoperability between tools.

b) RQ2. Tool support for fuzzy argumentation:: Existing tools are primarily oriented toward rational decision-making. In practice, when facing complex decisions, developers favour a more naturalistic approach. Tool support, as well as the modelling languages and techniques used for representing discussions, should reflect this practical need. The modelling of discussions is a difficult problem in itself in the field of computational linguistics where various approaches have been developed to represent and analyze them (e.g., [ 46 ], [ 47 ], [ 48 ]). However, combining such approaches with MDE techniques, languages and tools is an open problem. One potential avenue is to extend peliminary work on creating models of software design discussions [ 49 ], [ 50 ] with techniques from design space modelling and exploration. We are currently working on a project to model online discussions and provide online feedback and analytics about them to the participants. c) RQ3. Evolving decision models:: It is currently possible to make static argumentation models. It should be possible to take multiple snapshots of static models in order to reason on the evolution of the models over time. For reasoning on design decision rationale, the monitoring and management of compliance [ 51 ] and assurance [ 52 ] documentation over time is relevant. Crucially, decision-making should be informed by the principled evaluation of alternatives. This can be accomplished by integrating into CBSD techniques developed for design space exploration [ 53 ]. Further, a support tool should integrate these in a metamodelling platform that would allow comprehensive management [ 54 ] and versioning [ 55 ].

d) RQ4. Monitoring Team Dynamics:: Current tools provide limited information on team characteristics, like expertise (D2) or organizational context (D7). Since these characteristics vary from time to time, and from decision to decision, it would be important to keep track of them in order to support better reasoning. For example, CBCs might accrue “consensus debt” (akin to technical debt [ 56 ]) when provisional short term decisions need to be made to address urgent contingencies. This debt should be recorded, monitored and resolved. Keeping track of team dynamics would enable better modelling of the social network, which would connect to decision models with specific heuristics to detect degenerations. In this, it is crucial to integrate into CBSD insights from industrial psychology [ 57 ], taking into account the specific philosophical concerns of CBCs, especially centred around issues of intersectionality [ 58 ].

B. Epistemological approach

CBCs have their roots in social movements (e.g., Loomio originates from the Occupy Wall Street movement [ 38 ]), and research in CBSD should take in account its origins and the particular concerns that lie at its core philosophy. Specifically, the philosophy behind CBSD evolved from critiques of abuses of traditional organizational structures. From the viewpoint of social movements, inequalities extant in society are reproduced as individual behavior inside organizations unless special care is given to avoid them.

This includes, from the perspective of CBCs, academic researchers. This poses a practical and epistemological challenge for research in CBSD, since traditional post-positivist and constructivist epistemological stances might be met with significant resistance and suspicion from CBCs. Instead, we propose that critical theory is the appropriate epistemological framework for research in this domain. Easterbrook et al. describe it as follows [ 59 ]:

Critical Theory judges scientific knowledge by its ability to free people from restrictive systems of thought [ 60 ]. Critical theorists argue that research is a political act, because knowledge empowers different groups within society, or entrenches existing power structures

Within a critical-theoretic epistemological framework, researchers prioritize the use of participatory approaches, that provide them with intimate insights and high levels of familiarity with the teams being studied [ 61 ], as well as action research. Action research is a research methodology, where researchers acknowledge that they are not objective observers and instead work together with a community to affect change, explicitly acknowledging their biases. The focus is on building relationships of mutual trust and accountability between researchers and the communities they study, and on participatory theory building in order to address real problems withing the community, under the principle that research and action are indivisible [ 62 ].

In short, research on CBSD should be conducted in close collaboration with CBCs themselves, ensuring that their real needs are addressed in accordance to the specific concerns and priorities in their social and political milieu.

VII. RELATED WORK

The research questions provide interesting new avenues for the development of tool support for GDM within generic CBCs. However, how does this can be mapped to CBSD? Will the theoretical tool presented herein provide any benefit for software engineering, at least within CBC contexts?

The work of [ 6 ] on Agile software development corroborates some of the CBC challenges. They are critical of traditional decision theory being too rooted within the rational decision-making paradigm (RQ2):

Normative decision theory views decision makers as idealized, rational, extremely intelligent beings who overcome their inner turmoil, shifting values, anxieties, post-decision regrets, fear of ambiguity, inability to perform intricate calculations and limited attention span to make rational, optimum choices. [ 6 ]

They also see the self-managed nature of Agile software development as challenging. They mention that traditional authoritarian structures normally ensure that the voice of the experts carry more weight than the voice of others (RQ4). They also mention that in Agile, decision are not always implemented as discussed during meetings, if at all (UC5).

In practice however, Agile software development is not as self-managed as theoretically designed. [ 8 ], in a survey of Agile software developers, found that 67% of them still had a manager above them. These Agile teams come from traditional authoritarian organizations, or are within mixed organizations (Agile and non-Agile), and are still beholden to high management.

The works of [ 7 ], while not specific to software, does study engineering teams. They note the heterogeneity of experts (D1) within modern engineering teams. For example, a software development team might include a user interface expert, a database expert, a security expert, etc. They therefore build a theoretical model to help engineering team manage this heterogeneity of expertise, through a weighting approach (RQ4). This heterogeneity becomes especially difficult during software requirement negotiations, as it includes participants with varying backgrounds: domain experts, software developers, non-technical stakeholders, etc. [ 63 ]. [ 7 ]’s model integrates a feedback mechanism which recognizes that consensus is a building, evolving process (RQ3). The objective of their model is to support this evolving process by providing appropriate feedback to the participants, based on their expertise. As the authors write, engineering GDM “is a dynamic and iterative process, composed of several rounds where the experts express, discuss, and modify their preferences” [ 7 ].

The importance of expertise must however be balanced with the concern of technocratic autocracy, “where expertise is the basis of authority” [ 24 ]. The works of [ 24 ] outlines the importance of managing concerns (RQ5) to avoid the case where the decision-making process becomes the domain of unchallenged experts. Their work also show the lack of research into naturalistic decision-making in Agile software development. They write that:

Today’s software systems are becoming more complex because of the need to balance the often disparate needs of diverse stakeholders, and the growing complexity of the technology used. This reality requires more unstructured decision-making. Second, the principles of agile software development align with the definition of naturalistic decisionmaking. [ 24 ] There is therefore a need, at least within Agile software development, for naturalistic GDM support tools (RQ2).

In summary, the research questions identified are not generic needs of CBCs, but can also be applied to software engineering. A number of these research questions are not only limited to CBSD, but could also provide benefits for Agile software development. For Agile contexts, we note especially the need for naturalistic GDM tool support (RQ2) and the need for expertise management (RQ4).

VIII. CONCLUSION

We have outlined the group decision-making (GDM) process and its associated challenges in the context of ConsensusBased Software Development (CBSD), based on existing practices within Consensus-Based Communities (CBCs). We structured these in high-level requirements for supporting the CBSD with appropriate tooling and analyzed a preliminary list of CBSD-related tools to find gaps in the existing functionalities. Based on this analysis, we posed outlined a vision for research in CBSD, identigying four research directions and outlining the epistemological and methodological challenges for such research.

References

[1] T. Flanagan , C. Eckert , and P. J. Clarkson , “ Externalizing tacit overview knowledge: A model-based approach to supporting design teams,” Artificial Intelligence for Engineering Design, Analysis and Manufacturing , vol. 21 , no. 3 , pp. 227 - 242 , Jun. 2007 . [Online]. Available: http://dx.doi.org/10.1017/S089006040700025X

[2] M. J. Turk , “ Scaling a code in the human dimension,” in Proceedings of the Conference on Extreme Science and Engineering Discovery Environment: Gateway to Discovery, ser . XSEDE '13 . New York, NY, USA: ACM, 2013 , pp. 691 - 697 . [Online]. Available: http://doi.acm. org/10 .1145/2484762.2484782

[3] M. E. Conway , “ How do committees invent?” Datamation , vol. 14 , no. 4 , pp. 28 - 31 , 1968 .

[4] N. Nagappan , B. Murphy , and V. Basili , “ The influence of organizational structure on software quality: An empirical case study,” in Proceedings of the 30th International Conference on Software Engineering , ser. ICSE '08 . New York, NY, USA: ACM, 2008 , pp. 521 - 530 . [Online]. Available: http://doi.acm. org/10 .1145/1368088.1368160

[5] A. Maturo and A. G. S. Ventre , “ Models for consensus in multiperson decision making,” in NAFIPS 2008 - 2008 Annual Meeting of the North American Fuzzy Information Processing Society , May 2008 , pp. 1 - 4 .

[6] M. Drury , K. Conboy , and K. Power , “ Decision making in agile development: A focus group study of decisions and obstacles ,” in 2011 Agile Conference, Aug 2011 , pp. 39 - 47 .

[7] I. J . Pe´rez, F. J. Cabrerizo , S. Alonso , and E. Herrera-Viedma , “A new consensus model for group decision making problems with nonhomogeneous experts , ” IEEE Transactions on Systems, Man and Cybernetics: Systems , vol. 44 , no. 4 , pp. 494 - 498 , 2014 .

[8] Y. Shastri , R. Hoda , and R. Amor , “ Does the 'project manager' still exist in agile software development projects?” in 2016 23rd Asia-Pacific Software Engineering Conference (APSEC), Dec 2016 , pp. 57 - 64 .

[9] K. Beck , M. Beedle , A. van Bennekum , A. Cockburn , W. Cunningham , M. Fowler , J. Grenning , J. Highsmith , A. Hunt , R. Jeffries , J. Kern , B. Marick , R. C. Martin , S. Mellor , K. Schwaber , J. Sutherland , and D. Thomas. ( 2001 ) Principles behind the agile manifesto . [Online]. Available: http://agilemanifesto.org/principles.html

[10] Rishipal , “ Analytical comparison of flat and vertical organizational structure,” European Journal of Business and Management , vol. 6 , no. 36 , pp. 56 - 65 , 2014 .

[11] B. J. Robertson , “ Organization at the leading edge: Introducing holacracy , ” Tech. Rep. , 2007 .

[12] Koumbit . ( 2017 ) About us . [Online]. Available: https://www.koumbit. org/en/about

[13] Loomio Cooperative Limited. ( 2016 ) The loomio co-op handbook . [Online]. Available: https://www.loomio.coop/

[14] Zappos Insights. ( 2017 ) Holacracy - flattening the organization structure and busting bureaucracy . [Online]. Available: https://www. zapposinsights.com/about/holacracy

[15] A. Doyle. ( 2016 ) Management and organization at medium . [Online]. Available: https://blog.medium.com/ management-and - organization - at-medium-2228cc9d93e9

[16] Drupal . ( 2016 ) Drupal code of conduct . [Online]. Available: https: //www.drupal.org/dcoc

[17] Seeds for Change, A Consensus Handbook . Footprint Workers' Co-op, 2013 .

[18] D. G. Messerschmitt , C. Szyperski et al., “ Software ecosystem: understanding an indispensable technology and industry ,” MIT Press Books, vol. 1 , 2005 .

[19] D. Zwieback , Beyond Blame , Learning from successes and failure . O'Reilly Media , 2015 .

[20] A. Cornell , Oppose and Propose! AK Press, 2011 .

[21] D. Vannucci and R. Singer , Come Hell or High Water . AK Press, 2010 .

[22] R. Cialdini , “The science of persuasion,” Scientific American , vol. 284 , pp. 76 - 81 , 2001 .

[23] F. J. Cabrerizo , F. Chiclana , M. R. Ure n˝a, and E. Herrera-Viedma , “Challenges and open questions in soft consensus models,” in 2013 Joint IFSA World Congress and NAFIPS Annual Meeting (IFSA/NAFIPS ), June 2013 , pp. 944 - 949 .

[24] N. B. Moe , A. Aurum , and T. Dyba˚, “ Challenges of shared decisionmaking: A multiple case study of agile software development , ” Information and Software Technology , vol. 54 , no. 8 , pp. 853 - 865 , 2012 , special Issue: Voice of the Editorial Board .

[25] C. Zannier , M. Chiasson , and F. Maurer , “ A model of design decision making based on empirical results of interviews with software designers , ” Information and Software Technology , vol. 49 , no. 6 , pp. 637 - 653 , 2007 .

[26] M. Van Mechelen , M. Gielen , V. vanden Abeele , A. Laenen , and B. Zaman , “ Exploring challenging group dynamics in participatory design with children , ” in Proceedings of the 2014 Conference on Interaction Design and Children , ser. IDC ' 14 . New York, NY, USA: ACM, 2014 , pp. 269 - 272 . [Online]. Available: http: //doi.acm. org/10 .1145/2593968.2610469

[27] A. Kalyanasundaram , W. Wei , K. M. Carley , and J. D. Herbsleb , “ An agent-based model of edit wars in wikipedia: How and when is consensus reached,” in Proceedings of the 2015 Winter Simulation Conference, ser. WSC '15 . Piscataway , NJ, USA: IEEE Press, 2015 , pp. 276 - 287 . [Online]. Available: http://dl.acm.org/citation.cfm? id= 2888619 . 2888648

[28] P. Bourdieu , Distinction. Harvard University Press, 1984 .

[29] M. Shahin , P. Liang , and M. R. Khayyambashi , “ Architectural design decision: Existing models and tools,” in 2009 Joint Working IEEE/IFIP Conference on Software Architecture European Conference on Software Architecture , Sept 2009 , pp. 293 - 296 .

[30] J. Tyree and A. Akerman , “ Architecture decisions: demystifying architecture , ” IEEE Software , vol. 22 , no. 2 , pp. 19 - 27 , March 2005 .

[31] E. S. K. Yu , “ Towards modelling and reasoning support for earlyphase requirements engineering ,” in Requirements Engineering, 1997 ., Proceedings of the Third IEEE International Symposium on, Jan 1997 , pp. 226 - 235 .

[32] F. Dalpiaz , X. Franch , and J. Horkoff , “iStar 2.0 Language Guide,” Tech. Rep. , 2016 .

[33] D. A. L. Garc ´ıa, T. d. J. Mateo Sanguino, E. C. Ancos , and I. F . de Viana Gonza´lez, “ A debate and decision-making tool for enhanced learning , ” IEEE Transactions on Learning Technologies , vol. 9 , no. 3 , pp. 205 - 216 , July 2016 .

[34] G. Betz , S. Cacean , and C. Voigt. ( 2016 ) Argunet - open-source argument mapping . [Online]. Available: http://www.argunet.org/editor

[35] T. F. Gordon . ( 2016 ) Carneades - tools for argument (re)construction, evaluation, mapping and interchange . [Online]. Available: http: //carneades.github.io/

[36] ReasoningLab.com. ( 2017 ) bcisive - online decision mapping. [Online] . Available: https://www.bcisiveonline.com/

[37] P. Baldwin and D. Price. ( 2017 ) Debategraph . [Online]. Available: http://debategraph.org/

[38] Loomio Cooperative Limited. ( 2016 ) Loomio - better decisions together . [Online]. Available: https://www.loomio.org/

[39] Mountain Goat Software. ( 2017 ) Planningpoker.com - estimates made easy. sprints made simple . [Online]. Available: https://www. planningpoker.com/

[40] Reddit . ( 2017 ) Reddit . [Online]. Available: https://about.reddit.com/

[41] Civilized Discourse Construction Kit. ( 2017 ) What is discourse? - discourse - civilized discussion . [Online]. Available: http://www. discourse.org/about

[42] holaSpirit SAS . (2017) holaspirit - application for holacracy and teal organizations . [Online]. Available: https://www.holaspirit.com/

[43] J. Janssen , M. De Cock , and D. Vermeir , “ Fuzzy argumentation networks,” Vrije Universiteit Brussel , Tech. Rep ., January 2008 .

[44] C. Edwards , “Agreeing to fail,” Engineering & Technology, vol. 4 , no. 7 , pp. 74 - 75 , 2009 . [Online]. Available: https://doi.org/10.1049/et. 2009 .0716

[45] B. Combemale , J. Deantoni , B. Baudry , R. B. France, J.-M. Je ´ze´quel, and J. Gray , “Globalizing modeling languages,” Computer, pp. 10 - 13 , 2014 .

[46] D. P. Twitchell , M. Adkins , J. F. Nunamaker , and J. K. Burgoon , “ Using speech act theory to model conversations for automated classification and retrieval,” in Proceedings of the International Working Conference Language Action Perspective Communication Modelling (LAP 2004 ), 2004 , pp. 121 - 130 .

[47] S. Joty and E. Hoque, “ Speech act modeling of written asynchronous conversations with task-specific embeddings and conditional structured models,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics , ACL , 2016 , pp. 7 - 12 .

[48] R. P. Fawcett and B. L. Davies , Monologue as a turn in dialogue: Towards an integration of exchange structure and rhetorical structure theory , 1992 , pp. 151 - 166 .

[49] E. Black , P. McBurney , and S. Zschaler , Towards Agent Dialogue as a Tool for Capturing Software Design Discussions , 2014 , pp. 95 - 110 .

[50] A. S. Mashiyat , M. Famelis , R. Salay , and M. Chechik , “ Using developer conversations to resolve uncertainty in software development: A position paper ,” in Proceedings of the 4th International Workshop on Recommendation Systems for Software Engineering , 2014 , pp. 1 - 5 .

[51] S. Ghanavati , D. Amyot , and L. Peyton, “ Towards a framework for tracking legal compliance in healthcare,” in Advanced Information Systems Engineering . Springer, 2007 , pp. 218 - 232 .

[52] S. Kokaly , R. Salay , V. Cassano , T. Maibaum , and M. Chechik , “ A model management approach for assurance case reuse due to system evolution,” in Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems. ACM , 2016 , pp. 196 - 206 .

[53] S. A. Busari and E. Letier, “ Radar: A lightweight tool for requirements and architecture decision analysis,” in Proceedings of the 39th International Conference on Software Engineering , ser. ICSE '17 . Piscataway , NJ, USA: IEEE Press, 2017 , pp. 552 - 562 . [Online]. Available: https://doi.org/10.1109/ICSE. 2017 .57

[54] G. Brunet , M. Chechik , S. Easterbrook , S. Nejati , N. Niu , and M. Sabetzadeh , “ A manifesto for model merging ,” in Proceedings of the 2006 international workshop on Global integrated model management. ACM , 2006 , pp. 5 - 12 .

[55] P. Brosch , G. Kappel, P. Langer , M. Seidl , K. Wieland , and M. Wimmer , “ An introduction to model versioning,” in Proceedings of the 12th international conference on Formal Methods for the Design of Computer, Communication, and Software Systems: formal methods for modeldriven engineering . Springer-Verlag, 2012 , pp. 336 - 398 .

[56] P. Kruchten , R. L. Nord , and I. Ozkaya , “ Technical debt: From metaphor to theory and practice,” Ieee software , vol. 29 , no. 6 , pp. 18 - 21 , 2012 .

[57] K. W. Buffinton , K. W. Jablokow , and K. A. Martin , “ Project team dynamics and cognitive style,” Engineering Management Journal , vol. 14 , no. 3 , pp. 25 - 33 , 2002 .

[58] D. Atewologun , R. Sealy , and S. Vinnicombe , “ Revealing intersectional dynamics in organizations: Introducing 'intersectional identity work',” Gender , Work & Organization, vol. 23 , no. 3 , pp. 223 - 247 , 2016 .

[59] S. Easterbrook , J. Singer , M. - A. Storey , and D. Damian , “ Selecting empirical methods for software engineering research,” Guide to advanced empirical software engineering , pp. 285 - 311 , 2008 .

[60] C. Calhoun , Critical social theory: Culture, history, and the challenge of difference . Wiley-Blackwell, 1995 .

[61] T. C. Lethbridge , S. E. Sim , and J. Singer , “ Studying software engineers: Data collection techniques for software field studies,” Empirical Software Engineering , vol. 10 , no. 3 , pp. 311 - 341 , 2005 . [Online]. Available: http://dx.doi.org/10.1007/s10664-005-1290-x

[62] P. S. M. Dos Santos , G. H. Travassos , and M. V. Zelkowitz , “ Action research can swing the balance in experimental software engineering,” Advances in computers , vol. 83 , pp. 205 - 276 , 2011 .

[63] H. In and S. Roy, “ Issues of visualized conflict resolution ,” in Proceedings Fifth IEEE International Symposium on Requirements Engineering , 2001 , pp. 314 - 315 .