Supporting Consensus-based Software Development: a Vision Paper Mathieu Lavallée Guillaume Beaulieu Michalis Famelis Independent consultant Université du Québec à Montréal Université de Montréal math.lavallee@gmail.com beaulieu.guillaume.2@courrier.uqam.ca famelis@iro.umontreal.ca Abstract—Traditional, vertical organizational models of soft- More More horizontal vertical ware development have been challenged by more agile and col- laborative structures. Recently, this has also been demonstrated in the emergence of explicitly horizontalist organizational struc- Consensus-based Agile Traditional tures, focused on consensus-based decision making. In this paper, communities (CBCs) "Flat" organizations management Holarchies we describe the principles and processes of these “Consensus- Based Communities” (CBCs) and outline the main challenges Fig. 1: The continuum from vertical (i.e. hierarchical) to horizontal they face as they try to implement “Consensus-Based Software (i.e. non-hierarchical) organizations. Development” (CBSD). We express these as early, high level requirements for a tool supported methodology. Based on these, we present and analysis of existing tools that shows that no an “holarchy” [11], where decisions are delegated to self- single tool provides complete support for consensus-based group managing teams. Often, organizations using self-managed decision making. We thus outline directions for future research, identifying opportunities for the development and deployement teams still maintain a higher authority, simply reducing the of model-based techniques in this emergent field. amount of middle management. In Figure 1 we position Agile teams, that promote self-organized GDM, in the middle of the I. I NTRODUCTION continuum, along with flat and holarchic organizations. Software development is a team activity [1], [2], in which In this paper, we put the spotlight on the other end of the internal team dynamics impact the quality of the produced continuum. This is populated by software development orga- software. A classic formulation of this is “Conway’s Law”: nizations that intentionally adopt explicitly anti-hierarchical Any organization that designs a system (defined and consensus-based modes of GDM and collaboration. For broadly) will produce a design whose structure is a example, Koumbit, a web-hosting company based in Mon- copy of the organization’s communication structure. treal, describes its internal processes as “non-hierarchical self- [3] management” [12]. Loomio is a developer-owned cooperative More recently, Nagappan et al. showed that organizational based in New Zealand that puts consent at the centre of its structure was a better predictor of fault-proneness than tra- decision making process [13]. Other organizations like Zappos ditional technical metrics [4]. A defining aspect of a team’s [14] and Medium.com [15] maintain horizontal principles. organization and communication structure is Group Decision This is also often the case for open source communities, Making (GDM), also known as Multi-Person Decision-Making such as Drupal, which favours a self-managed approach [16]. (MPDM), [5], [6], [7]. Traditionally, organization structures We call such organizations Consensus-Based Communities are hierarchical and command based. However, the advent of (CBCs). Agile software development has changed GDM processes, as CBCs face a set of unique challenges when developing it tends to prefer horizontal structures [8]. Characteristically, software, mostly based on their philosophical approach to the Agile Manifesto declares: GDM. The Agile Manifesto already pushed GDM away from The best architectures, requirements, and designs the “Benevolent Dictator” management paradigm. emerge from self-organizing teams. [9] Agile Software Development has completely dis- In practice, the degree of self-organization varies widely pensed with the formal job title of the project man- across different organizations. We illustrate this diversity in ager. Agile methodologies such as scrum seem to Figure 1, as a continuum of organizational horizontality in distribute the erstwhile responsibilities of the project software development teams. On the one end there is tra- manager into new roles. [8] ditional “command-based” development, where upper levels CBCs affirm that team decisions must go a step further, and of the organizational hierarchy have the final say on what ensure consensus building as much as possible, instead of developers do. Moving from that extreme, some organizations majority decision-making. The rationale behind striving for maintain the use of project managers who have the final say in consensus is based on a perceived flaw of democratic decision- decisions while encouraging the building of a team consensus. making called “tyranny of the majority”: Some organizations actively pursue a “flat” model [10] or The will of the majority may be seen as the will of the whole group, with the minority expected to point out areas for improvement in Section VI. We discuss accept and carry out the decision, even if it is against related work in Section VII and conclude in Section VIII. their deeply held convictions and most basic needs. It is possible for a voting group to look for solutions II. BACKGROUND that would suit everybody, but it is more common for To better understand the unique challenges in CBSD, we ideas with a majority backing to be pushed through. first outline the main characteristics of CBCs. [17, p. 9] Consensus-based decision-making does not mean that ev- This flaw can cause, for example a team with a majority of erybody must agree on everything. In practice, consensus non-experts to overrule a minority of experts, leading to poor is typically “soft”, which means that certain forms of dis- decisions. Instead, consensus-building requires team members agreement are acceptable. According to the Consensus Hand- to convince all their colleagues, instead of unilaterally impos- book [17, p. 6]: ing their propositions. This means that software developers in Instead of simply voting for an item and having CBCs cannot exploit the process in order to shut down their the majority of the group get their way, a consensus colleagues, but must provide facts and data able to persuade group is committed to finding solutions that every- them to change their mind. one actively supports, or at least can live with. CBCs such as the ones mentioned above also operate under anti-hierarchical principles. Their perception is that software The aim is to avoid a situation where there are winners and development has become so complex that not one person can losers. CBCs claim that this helps everybody feel involved in see the whole picture. Each software developer is therefore the organization, and helps them learn new skills and be better, a piece of the puzzle, each with their own expertise, and more complete software developers [12]. This further increases decisions should then be made with everyone in mind [7]. the sense of agency and ownership towards decisions, since Even though CBCs are currently not the norm in software each individual actively participates, ensuring that everyone’s engineering, studying them can provide interesting insights on concerns are part of the process. A major benefit of consensus team dynamics, and can help identify beneficial organizational decision-making is therefore the increased solidarity among practices across the continuum of organizational horizontality. team members in cases where bad decisions are made. Faced Broadly speaking, the recent move away from traditional with a negative result, since the whole team was engaged in organizational practices can be understood as a concern about the decision together, no single individual or group can be the degree to which they are appropriate for software de- singled out for blame [19]. velopment. Thus, interesting insights can be obtained from In the following, we describe a typical consensus-based radically different organizational structures [10]. At the same GDM process, in terms of the activities performed and time, since CBCs are software producing organizations in an the roles and responsibilities associated. Consensus-based era of software ecosystems [18], understanding the impact of decision-making processes are adapted from the different kinds their practices on software quality (cf. Conway’s Law) may of democratic practices surrounding them [20]. They are have ramifications beyond their niche. adapted to the size and ability of the group. In this paper, we focus on CBCs and propose a vision While the exact process varies from one organization to for a comprehensive, model-based, tool supported method- another, we present here a basic model of typical activities in ology for Consensus-Based Software Development (CBSD) consensus-based GDM [17, p. 16]. These are: that addresses their unique set of challenges. We identify Introduction (A1): Introduction of the issue on which a the specific needs of CBCs and study existing collaboration decision must be made; tools to find gaps between the two and potential avenues of Discussion (A2): Discuss the issue and collect ideas; research and development. Specifically, we make the following Concerns (A3): List concerns regarding the proposal; contributions: Proposal (A4): Identify emerging proposals meeting con- a) We highlight the existence of CBCs and describe their cerns; unique processes and challenges; Refinement (A5): Discuss, clarify and amend the proposals; b) We describe how consensus-based GDM can be sup- Consensus test (A6): Test for agreement, usually through a ported for software development; straw poll. If not, return to the previous points; c) We analyze the capabilities of existing tools, identify their Implementation (A7): Implement the decision. limitations, and outline how model-based techniques can We show these activities diagrammatically in Figure 2. be used to improve the state of CBSD practise. During a test for consensus (activity A6), a typical CBC The rest of the paper is organized as follows: We present uses four types of votes for the straw poll [17, p. 16] with the background information on CBCs in Section II, detailing following meaning: their typical internal processes In Section III, we explain the Block (V1): The participant thinks the proposal is funda- challenges raised by consensus-based GDM and in Section IV mentally wrong and should not be implemented by the we outline the requirements for a tool framework for support- organization. ing consensus-based software development. We analyze the Stand aside (V2): The participant cannot support the pro- capabilities of existing GDM support tools in Section V and posal and is not willing to implement it. However they R1. Facilitator includes the minutes of the meeting, along with the decisions R4. Participant and action items. It is recommended that action items be A1. Introduction tracked independently from the rest in order to avoid losing R3. Hand takers track of them and to prevent power concentration into the same Issues F1. hands [21]. to adress A2. Discussion Active listening F2. ? Summarising A3. Concerns III. C HALLENGES OF C ONSENSUS -BASED GDM A4. Proposal F3. Software development teams that adopt a consensus-based Synthesis V1. Block GDM process face a unique set of challenges. These stem from A5. Refinement the need to work and reach consensus. Degenerations of the process can introduce problems such as false consensus, where V2. Stand aside Issues Consensus raised A6. Consensus test an explicit agreement is reached, but where team members V3. Reservation reached ? ? implicitly disagree. We group such challenges along seven different dimensions. Acceptable A7. Implementation V4. Agreement One dimension (D1) is related to the heterogeneity of issues communication [7]. For example, software engineering teams ? R2. Minute takers include a variety of experts of different fields, depending on Fig. 2: Consensus decision making model. the needs of the project. These different experts can be profi- cient in different languages and techniques. For example, one expert can be well-versed with the UML language while the are willing to let the organization implement it without rest of the team is not. This creates a hurdle for communication them. as the expert struggles to explain his point to non-experts. Reservation (V3): The participant has some reservations, but Another dimension (D2) is related to the need to persuade is willing to let the proposal be implemented and to work others in order to obtain consensus. Ideally, this persuasion is for its implementation. based on real facts, brought by real experts, but in practice, Agreement (V4): The participant supports the proposal and emotional factors can enter the equation. For example, “people is willing to implement it. are easily persuaded by other people that they like” ([22], The responsibilities associated with the good functioning quoted by [23]). Another example, “consistently slandering of the team are traditionally assigned to a team manager. someone behind his back” [21, p. 33] is another tactic to Instead, in self-managing teams, they must be delegated to persuade others using unsavory means. team members. Consensus-based teams recommend to split This leads to the third dimension (D3) of how to ade- responsibilities to different people to avoid a concentration of quatly generate alternatives for a complex issue. Ideally, GDM powers which would re-create a hierarchical structure [21]. follows a rational decision-making approach. This approach Specific roles include [17], [21]: is akin to a breadth-first approach, where all alternatives Facilitators (R1): A neutral party whose responsibility is to are studied and compared. The opposite is the naturalistic keep the meeting on topic and assist the decision-making decision-making approach. This approach is akin of a depth- process. In some cases, no single person is assigned the first approach, where one easy to find solution is studied in- role, instead relying on the participants, assuming that depth [24]. For example, a typical software development team they have internalized it enough to self-discipline. mixes the two approaches, going in-depth with a small number Minute Takers (R2): Responsible for keeping track of the of alternatives [25]. Generally this is because the complexity of decisions and action items. software development context makes a breadth-first approach Hand Takers (R3): Responsible of keeping track of whose impossible and requires limiting the breadth to what seems turn it is to speak next and control the time allotted to possible at the time [24], [25]. each speaker. Generating and choosing the proper alternatives can be Participant (R4): Each team member is responsible for self- difficult, especially with dysfunctional teams (D4). Issues of discipline and for the good functioning of the process. “groupthink”, where there is a reluctance of criticizing ideas, Facilitation is a complex role, which can be broken down are well documented [26]. Instances of polarized factions into the following responsibilities [17, p. 31]: constantly blocking the propositions of the others are also Active listening (F1): To assist speakers in expressing their a symptom of a dysfunctional team [26], [27]. Even before ideas and its associated context. such problems arise, social stratification due to factors such Summarising (F2): To reformulate the intervention of the as racism, sexism and class, generally makes marginalized speaker in a clear and concise manner in order to confirm people take less share in the decision-making process. Worse, its interpretation. that phenomenon is often justified by those who are silencing Synthesis (F3): To merge the ideas in actual proposals po- themselves as they perceives themselves as less competent tentially acceptable for all participants. [28]. Creating a context for genuine participation is an ongoing The list of work products of a consensus-based process process that requires patience and dedication. D2. D1. Evaluate Manage the difference and expertise A1. and of languages Introduce the issue UC7. Identify expertise A2. and UC6. UC1. Support whiteboarding A7. Discuss the issue helps Implement the Present the issue helps and and helps decision A3. UC5. List concerns and helps Supporting Monitor decisions UC2. D5. helps A4. and Manage the proposals the CBSD and Self-monitor Keep track of proposals process decisions and helps A5. Refine proposals helps and UC4. UC3. Detect consensus Document the arguments and R2. and Take the minutes and and A6. Test for consensus R3. F3. and and and and Manage speakers Synthesize V1. V2. V3. V4. interventions Record Record stand Record Record block votes aside votes reservation votes agreement votes Fig. 3: Use-cases to support for the CBSD process, using the i* modeling language. The fifth dimension is related to the nature of anti- to the letter, but changes are actually frequent: Since there hierarchical organizations (D5). Without a manager, the de- are no leader to impose a process, any meeting can result velopers themselves must assume responsibility of the task in a process change. Hence, the bulk of the decision-making typically assigned to managers. These include managing con- process is centred around educating participants on the current flicting priorities between stakeholders, assuring that action organizational context. Documentation of such processes is items are actually implemented, and giving more weight to contextual and reflects a community’s ongoing organizational the intervention of experts [6], [10]. dynamics. Therefore, the given organizational context of the Another challenge is related to documenting the rationale CBC (D7) will have an impact on its GDM, more so than of the decisions made (D6). As [29] writes: in a traditional contexts using common GDM practices. The rationale of the decision (D6) cannot be understood outside In the field of software architecture, there has been a this constantly evolving organizational context of the CBC paradigm shift from describing the outcome of archi- (D7). tecting process mostly described by component and connector (know-what) to documenting architectural IV. R EQUIREMENTS FOR CBSD SUPPORT TOOLS design decisions and their rationale (know-how) We identify the high-level requirements for CBSD tools, which leads to the production of an architecture. [29] based on the processes, activities, roles, and artifacts involved A GDM support tool should therefore document decision in CBSD described in Section II, as well as the challenges rationale to ensure that future developers can understand what outlined in Section III. We present the requirements in the arguments supported the decision. Ideally, such a tool would be form of use-cases, listed below. We also give the requirements able to see how decisions evolve [30]. For example, developers as goal models in Figures 3 and 4 using i*, a language for should know that a decision can be changed if the underlaying modelling early requirements [31], [32]. arguments are no longer valid. UC1: During the introduction of an issue, present its context But the technical context of the issue is not the only context in a concise yet understandable way (A1). to manage. Since CBCs are not the norm, the GDM process UC2: Collect the ideas brought during the meeting (A2), is generally a two-way educational process, first teaching obtain concerns from the participants and keep track of newcomers about the GDM process, but also teaching ways them (A3). Turn ideas into actionable proposals that ev- to address concerns in a constructive manner. With consensus- eryone can inspect and amend (A4). Record amendments based GDM, communities transition from a discursive space to proposals (A5). in which people can be either right or wrong to one where UC3: Document the arguments pertaining to each proposal people are either more or less aware of the issues surrounding (A5) and the identity of the person(s) who brought the its practices. To make things more complicated, as is the argument forward (R3). While it would be difficult to case for any organization, the typical process is not followed support active listening (F1) and the reformulation of V. W HAT IS C URRENTLY M ISSING IN CBSD S UPPORT Supporting D7. T OOLS ? CBSD Educate the organizational introspection context In this section we describe how existing tools and tech- and nologies fit with the CBSD activities, roles and work products helps helps described in previous sections. Ideally, CBSD support tools UC8. helps UC10. should be compatible with the processes presented earlier, and Evaluate alternatives Understand decisions address the challenges of consensus-based decision-making. UC9. In practice, while some are covered by existing tools, other and and Detect dysfunctions functionalities are still missing. The list of tools analyzed D6. is a preliminary, non-exhaustive list of tools, based on a D3. and Document Evaluate the breadth similar list published in the literature [33] as well as our own D4. rationale of the alternatives Identify groupthink and investigation. studied other dysfunctional ArguNet [34] and Carneades [35] are tools for structuring decision processes debates into arguments and logical propositions. They can Fig. 4: Use-cases to support for CBSD introspection, using the i* record the ideas and arguments presented during a GDM modeling language. meeting, using formal logic to encode their structure. They treat arguments as logically true propositions which can either support or oppose a proposal. They do not have voting mechanisms or support for evaluating expertise. ArguNet uses a directed node graph similar to concept models to link participants’ intervention (F2), the tool should assist the arguments and logicial propositions. Carneades, on the other Facilitator’s synthesis responsibility (F3). Each decision hand, uses a tree modeling langage similar to i* models in should include the minutes of the relevant meetings in order to structure arguments. order to understand better the context of the decision bCisive [36] is a tool for structuring debates. It provides (R2). a richer modelling language (over fifty pictograms) than UC4: Detect consensus, through mechanisms like straw polls ArguNet and Carneades. For example, it allows typing the (A6). Straw polls should support the four types of votes sources of arguments, e.g., to differentiate between expert (V1-V4). It should be possible to record the rationale of opinion and hearsay. Like Carneades, bCisive uses a tree non-consenting votes (V1-V3). modeling langage similar to i* models. UC5: Monitor the implementation of decisions after the meet- DebateGraph [37] is a mind-mapping tool offering the ca- ing, to ensure it is carried out as decided (A7, D5). pability to structure data. It can link arguments and proposals, UC6: Record relevant sketches (e.g., whiteboard photos). but its utility remains limited for GDM. Like ArguNet, it Discussion can often lead to a refined model that can uses a directed node graph similar to concept models to link be understood by all participants (D1). documents together. UC7: Identify and evaluate the expertise relevant to the deci- Loomio [38] is a GDM tool useful for voting when there are sion at hand (D2). already clearly-defined alternatives. It does not help in finding and structuring the debate in order to find these alternatives. It Introspection, reflection and self-improvement is a big part does not directly use a modeling language, but it uses simple of the Agile philosophy [9], and the philosophy of CBCs. A graphs (pie charts, bar charts) to display the level of consensus. CBSD support tool should also therefore support the following Planning Poker [39] is useful for choosing an appropriate use cases: priority or estimate. Alternatives are clearly stated in the form of numerical scales. The facilitator can choose among various UC8: After implementing a decision, determine if the alter- scales (such as the Fibonacci sequence, powers of 2, Small- natives studied were adequate or if the team should have Medium-Large, etc.) according to what is most appropriate for spent more time finding alternatives (D3). a given context. Planning poker uses a pictogrammic modeling UC9: Detect degenerated group decision making problems, langage based on common playing cards in order to facilitate such as groupthink, unresolved conflict, etc. (D4). understanding. UC10: Some decisions can have a long-term impact. Develop- Reddit [40] and Discourse [41] are discussion tools, where ers years later might have to deal with the decisions made participants bring arguments based on an introduction state- today and might want to understand why the specific ment. Reddit integrates a voting mechanism which ensures decision was made (D6). Understanding the decision that entries supported by more participants are ranked first. means understanding the organizational context wherein Discourse is closer to a mailing list and presents arguments in the decision was made at the time. Participants must a temporal sequence. While Reddit does not use a modelling therefore be educated on the organizational context in language, Discourse uses a simple pictogrammic model based order to understand the related decisions (D7). on users’ avatars in order to show the state of a discussion. A TABLE I: Results of the tool analysis based on the i* models in Figure 3 and 4. Tools are presented in alphabetical order. Tool Argunet bCisive Carneades DebateGraph Discourse holaSpirit Loomio Planning Poker Reddit UC1: Present issue A1 Yes Yes No No Yes No Yes Yes Yes A2 Yes Yes Yes Yes Yes No No No Yes A3 Partial Yes No Partial No No No No No UC2: Manage proposals A4 Yes Yes Yes No No No No No No A5 Partial Partial No No Partial No No No Partial A5 Yes Yes Yes Yes Yes No No No Yes UC3: Document R2 No No No No Yes No No No Yes arguments R3 No No No No Yes No No No Yes F3 Yes Yes Yes Yes No Yes No Yes No V1 No No No No No No Yes Yes Yes V2 No No No No No No Yes No No UC4: Detect consensus A6 V3 No No No No No No Yes No No V4 No No No No No No Yes Yes Yes A7 No No No No No No No Partial No UC5: Monitor decisions D5 No No No No No No No Partial No UC6: Whiteboarding D1 No Yes No Yes Yes No No No Partial UC7: Identify expertise D2 No Yes No No No Yes No No No UC8: Evaluate D3 No No No No No No No No No alternatives UC9: Detect D4 No No No No No Partial No No No dysfunctions UC10: Understand D6 Yes Yes Yes No Yes No No No Yes decisions D7 No No No No No No No No No Discourse discussion is modeled with the avatar of the initiator, different paradigm used by the tools analyzed. About half and avatars of the last users to reply to the discussion. the tools studied support a rational decision process (e.g. holaSpirit [42] is an implementation of the “holacracy” [11] ArguNet, bCisive, Carneades), rooted in logical propositions framework, which focuses on managing responsibilities and akin to Bayesian logic. These tools assume that arguments meeting agendas. It includes a way to document tensions brought forward are equally true and build the GDM around between participants, which can help diagnose some dys- argument structures. The other half support a naturalistic functionalities. It can help identify expertise, based on the decision process (e.g. Loomio, Reddit, Planning Poker) and assignment of responsibilities. The “holacracy” framework are rooted in fuzzy propositions. These fuzzy propositions are relies on Venn diagrams to show how responsibilities can be not assumed to be true, and GDM is therefore built around regrouped into roles. participants’ votes. We summarize the results of our analysis in Table I. Note The importance of both rational and naturalistic approaches that the activity “Refinement” (A5) appears in two columns to GDM has been detailed in published literature [24], [25]. On and does not present the same results. The reason is that one the one hand, purely rational tools are limited in practice. On analysis is taken from the point of view of management pro- the other hand, while some fuzzy argumentation models exist, posals (UC2) while the other is based on argument documenta- they focus on artificial intelligence and are not supported by tion (UC3). For example, a tool can permit edition of proposals any GDM tools [43]. This implies that a fuzzy argumentation (A5 from UC2), but does not cover argument documentationm modelling support tool might be useful for practical GDM, (A5 from UC3), therefore resulting in different results. and in extension, to CBSD. Studying the results presented in Table I, it is possible to Results show only partial coverage of the “Refinement” pinpoint incompatibilities between current tool functionalities activity (A5) from the point of view of proposal management and the needs of CBSD. Some requirements are poorly covered (UC2). A few tools support the editing of proposals and by current tools, while others are not covered at all. arguments, but none keep track of changes. This requirement Results also show that many tools focus on one aspect of is important in practice because it is often necessary to come GDM exclusively. This is coherent with the current tendency back to previous versions of proposals, e.g., when a change away from a single tool able to do everything (e.g., Rational is blocked by participants. It is also important for postmortem Rose) and toward multiple inter-communicating tools (e.g., evaluation of the GDM process, especially postmortem alter- integrating Slack with JIRA and Bitbucket). It would therefore native evaluation (UC8) and team dysfunction identification theoretically be possible to cover the needs of CBSD using (UC9). multiple inter-communicating tools. Unfortunately, most of Among the uncovered requirements, none of the analyzed these tools cannot do this right now, forcing a manual transfer tools support postmortem alternative evaluation (UC8). This is of data from one tool to the next. For example, while none cause for concern since poorly framed problems are the source of the tools analyzed provides direct support for decision of multiple software catastrophes [44]. It would be interesting monitoring (UC5), it would be possible to support it through a to track changes made to the alternatives throughout the project connection to a project management tool like Microsoft Project in order to detect when poor alternatives were chosen, or where or Atlassian JIRA. radical changes to alternatives occurred during development. However, one obstacle to this integration is the two widely No tools support the postmortem evaluation of team dys- functions (UC9), except holaSpirit, which does it partially c) RQ3. Evolving decision models:: It is currently possi- through the manual identification of tensions. With a tool able ble to make static argumentation models. It should be possible to keep track of changes, it would be possible to see how the to take multiple snapshots of static models in order to reason decision evolved throughout the project. It would therefore be on the evolution of the models over time. For reasoning on possible to perform some reasoning not only on one static design decision rationale, the monitoring and management of model, but on a serie of evolving models. It seems however compliance [51] and assurance [52] documentation over time that at this time, no framework exists to study the evolution is relevant. Crucially, decision-making should be informed by of argument models. the principled evaluation of alternatives. This can be accom- Given the importance given to the management of expertise plished by integrating into CBSD techniques developed for in the literature [7], [6], [44], it is surprising that more tools design space exploration [53]. Further, a support tool should are not taking it into account. Only two tools support it, but integrate these in a metamodelling platform that would allow they do not implement voting mechanisms. In other words, comprehensive management [54] and versioning [55]. expertise is not factored in any of the vote-based tools. This d) RQ4. Monitoring Team Dynamics:: Current tools pro- limits the capability of stakeholders to assess the relative vide limited information on team characteristics, like expertise merits of alternatives, as all proposals, arguments and votes (D2) or organizational context (D7). Since these characteristics are considered equal, which might not be the case in practice. vary from time to time, and from decision to decision, it would be important to keep track of them in order to support VI. G OING F ORWARD better reasoning. For example, CBCs might accrue “consensus Given the mismatch between the capabilities of existing debt” (akin to technical debt [56]) when provisional short term tools and the need of CBCs, we propose a roadmap for decisions need to be made to address urgent contingencies. research on model-based techniques for supporting CBSD. This debt should be recorded, monitored and resolved. Keep- We focus on model-based techniques specifically because the ing track of team dynamics would enable better modelling of Model Driven Engineering (MDE) approach allows describing the social network, which would connect to decision models the various relevant artifacts at a high level of abstraction, with specific heuristics to detect degenerations. In this, it which in turn helps integrate and analyze information from is crucial to integrate into CBSD insights from industrial heterogeneous sources, such as discussions, decisions, design psychology [57], taking into account the specific philosoph- artifacts, documentation, source code, etc. ical concerns of CBCs, especially centred around issues of intersectionality [58]. A. Main research questions Research on CBSD should at a minimum address the B. Epistemological approach following: CBCs have their roots in social movements (e.g., Loomio a) RQ1. Inter-tool communication:: Developers should originates from the Occupy Wall Street movement [38]), and be able to seamlessly use chains of relevant tools. For example, research in CBSD should take in account its origins and the they could pick one tool for problem statement, a second particular concerns that lie at its core philosophy. Specifically, for proposals and a third for argument documentation (UC1, the philosophy behind CBSD evolved from critiques of abuses UC2, UC3), and be able to link the decision data to a project of traditional organizational structures. From the viewpoint of management tool (UC5) of their choice. It is a classic problem social movements, inequalities extant in society are reproduced in software engineering and model-driven development. We as individual behavior inside organizations unless special care propose to use metamodelling and language globalization [45] is given to avoid them. to create an infrastructure for interoperability between tools. This includes, from the perspective of CBCs, academic b) RQ2. Tool support for fuzzy argumentation:: Existing researchers. This poses a practical and epistemological chal- tools are primarily oriented toward rational decision-making. lenge for research in CBSD, since traditional post-positivist In practice, when facing complex decisions, developers favour and constructivist epistemological stances might be met with a more naturalistic approach. Tool support, as well as the significant resistance and suspicion from CBCs. Instead, we modelling languages and techniques used for representing propose that critical theory is the appropriate epistemological discussions, should reflect this practical need. The modelling framework for research in this domain. Easterbrook et al. of discussions is a difficult problem in itself in the field of describe it as follows [59]: computational linguistics where various approaches have been developed to represent and analyze them (e.g., [46], [47], Critical Theory judges scientific knowledge by its [48]). However, combining such approaches with MDE tech- ability to free people from restrictive systems of niques, languages and tools is an open problem. One potential thought [60]. Critical theorists argue that research avenue is to extend peliminary work on creating models of is a political act, because knowledge empowers dif- software design discussions [49], [50] with techniques from ferent groups within society, or entrenches existing design space modelling and exploration. We are currently power structures working on a project to model online discussions and provide Within a critical-theoretic epistemological framework, re- online feedback and analytics about them to the participants. searchers prioritize the use of participatory approaches, that provide them with intimate insights and high levels of famil- non-technical stakeholders, etc. [63]. [7]’s model integrates iarity with the teams being studied [61], as well as action a feedback mechanism which recognizes that consensus is a research. Action research is a research methodology, where building, evolving process (RQ3). The objective of their model researchers acknowledge that they are not objective observers is to support this evolving process by providing appropriate and instead work together with a community to affect change, feedback to the participants, based on their expertise. As the explicitly acknowledging their biases. The focus is on building authors write, engineering GDM “is a dynamic and iterative relationships of mutual trust and accountability between re- process, composed of several rounds where the experts ex- searchers and the communities they study, and on participatory press, discuss, and modify their preferences” [7]. theory building in order to address real problems withing the The importance of expertise must however be balanced community, under the principle that research and action are with the concern of technocratic autocracy, “where expertise indivisible [62]. is the basis of authority” [24]. The works of [24] outlines In short, research on CBSD should be conducted in close the importance of managing concerns (RQ5) to avoid the collaboration with CBCs themselves, ensuring that their real case where the decision-making process becomes the domain needs are addressed in accordance to the specific concerns and of unchallenged experts. Their work also show the lack of priorities in their social and political milieu. research into naturalistic decision-making in Agile software development. They write that: VII. R ELATED W ORK Today’s software systems are becoming more com- The research questions provide interesting new avenues for plex because of the need to balance the often the development of tool support for GDM within generic disparate needs of diverse stakeholders, and the CBCs. However, how does this can be mapped to CBSD? growing complexity of the technology used. This Will the theoretical tool presented herein provide any benefit reality requires more unstructured decision-making. for software engineering, at least within CBC contexts? Second, the principles of agile software development The work of [6] on Agile software development corrob- align with the definition of naturalistic decision- orates some of the CBC challenges. They are critical of making. [24] traditional decision theory being too rooted within the rational decision-making paradigm (RQ2): There is therefore a need, at least within Agile software development, for naturalistic GDM support tools (RQ2). Normative decision theory views decision makers In summary, the research questions identified are not generic as idealized, rational, extremely intelligent beings needs of CBCs, but can also be applied to software engineer- who overcome their inner turmoil, shifting values, ing. A number of these research questions are not only limited anxieties, post-decision regrets, fear of ambiguity, to CBSD, but could also provide benefits for Agile software inability to perform intricate calculations and limited development. For Agile contexts, we note especially the need attention span to make rational, optimum choices. for naturalistic GDM tool support (RQ2) and the need for [6] expertise management (RQ4). They also see the self-managed nature of Agile software development as challenging. They mention that traditional VIII. C ONCLUSION authoritarian structures normally ensure that the voice of the We have outlined the group decision-making (GDM) pro- experts carry more weight than the voice of others (RQ4). cess and its associated challenges in the context of Consensus- They also mention that in Agile, decision are not always Based Software Development (CBSD), based on existing implemented as discussed during meetings, if at all (UC5). practices within Consensus-Based Communities (CBCs). We In practice however, Agile software development is not structured these in high-level requirements for supporting the as self-managed as theoretically designed. [8], in a survey CBSD with appropriate tooling and analyzed a preliminary list of Agile software developers, found that 67% of them still of CBSD-related tools to find gaps in the existing functional- had a manager above them. These Agile teams come from ities. Based on this analysis, we posed outlined a vision for traditional authoritarian organizations, or are within mixed research in CBSD, identigying four research directions and organizations (Agile and non-Agile), and are still beholden outlining the epistemological and methodological challenges to high management. for such research. The works of [7], while not specific to software, does study engineering teams. They note the heterogeneity of experts (D1) R EFERENCES within modern engineering teams. For example, a software [1] T. Flanagan, C. Eckert, and P. J. Clarkson, “Externalizing tacit development team might include a user interface expert, a overview knowledge: A model-based approach to supporting design database expert, a security expert, etc. They therefore build teams,” Artificial Intelligence for Engineering Design, Analysis and Manufacturing, vol. 21, no. 3, pp. 227–242, Jun. 2007. [Online]. a theoretical model to help engineering team manage this het- Available: http://dx.doi.org/10.1017/S089006040700025X erogeneity of expertise, through a weighting approach (RQ4). [2] M. J. Turk, “Scaling a code in the human dimension,” in This heterogeneity becomes especially difficult during soft- Proceedings of the Conference on Extreme Science and Engineering Discovery Environment: Gateway to Discovery, ser. XSEDE ’13. ware requirement negotiations, as it includes participants with New York, NY, USA: ACM, 2013, pp. 691–697. [Online]. Available: varying backgrounds: domain experts, software developers, http://doi.acm.org/10.1145/2484762.2484782 [3] M. E. Conway, “How do committees invent?” Datamation, vol. 14, no. 4, Conference, ser. WSC ’15. Piscataway, NJ, USA: IEEE Press, pp. 28–31, 1968. 2015, pp. 276–287. [Online]. Available: http://dl.acm.org/citation.cfm? [4] N. Nagappan, B. Murphy, and V. Basili, “The influence of organizational id=2888619.2888648 structure on software quality: An empirical case study,” in Proceedings [28] P. Bourdieu, Distinction. Harvard University Press, 1984. of the 30th International Conference on Software Engineering, ser. [29] M. Shahin, P. Liang, and M. R. Khayyambashi, “Architectural design ICSE ’08. New York, NY, USA: ACM, 2008, pp. 521–530. [Online]. decision: Existing models and tools,” in 2009 Joint Working IEEE/IFIP Available: http://doi.acm.org/10.1145/1368088.1368160 Conference on Software Architecture European Conference on Software [5] A. Maturo and A. G. S. Ventre, “Models for consensus in multiperson Architecture, Sept 2009, pp. 293–296. decision making,” in NAFIPS 2008 - 2008 Annual Meeting of the North [30] J. Tyree and A. Akerman, “Architecture decisions: demystifying archi- American Fuzzy Information Processing Society, May 2008, pp. 1–4. tecture,” IEEE Software, vol. 22, no. 2, pp. 19–27, March 2005. [6] M. Drury, K. Conboy, and K. Power, “Decision making in agile [31] E. S. K. Yu, “Towards modelling and reasoning support for early- development: A focus group study of decisions and obstacles,” in 2011 phase requirements engineering,” in Requirements Engineering, 1997., Agile Conference, Aug 2011, pp. 39–47. Proceedings of the Third IEEE International Symposium on, Jan 1997, [7] I. J. Pérez, F. J. Cabrerizo, S. Alonso, and E. Herrera-Viedma, “A pp. 226–235. new consensus model for group decision making problems with non- [32] F. Dalpiaz, X. Franch, and J. Horkoff, “iStar 2.0 Language Guide,” Tech. homogeneous experts,” IEEE Transactions on Systems, Man and Cyber- Rep., 2016. netics: Systems, vol. 44, no. 4, pp. 494–498, 2014. [33] D. A. L. Garcı́a, T. d. J. Mateo Sanguino, E. C. Ancos, and I. F. [8] Y. Shastri, R. Hoda, and R. Amor, “Does the ’project manager’ still de Viana González, “A debate and decision-making tool for enhanced exist in agile software development projects?” in 2016 23rd Asia-Pacific learning,” IEEE Transactions on Learning Technologies, vol. 9, no. 3, Software Engineering Conference (APSEC), Dec 2016, pp. 57–64. pp. 205–216, July 2016. [9] K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham, [34] G. Betz, S. Cacean, and C. Voigt. (2016) Argunet - open-source M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, J. Kern, argument mapping. [Online]. Available: http://www.argunet.org/editor B. Marick, R. C. Martin, S. Mellor, K. Schwaber, J. Sutherland, and [35] T. F. Gordon. (2016) Carneades - tools for argument (re)construction, D. Thomas. (2001) Principles behind the agile manifesto. [Online]. evaluation, mapping and interchange. [Online]. Available: http: Available: http://agilemanifesto.org/principles.html //carneades.github.io/ [10] Rishipal, “Analytical comparison of flat and vertical organizational [36] ReasoningLab.com. (2017) bcisive - online decision mapping. [Online]. structure,” European Journal of Business and Management, vol. 6, Available: https://www.bcisiveonline.com/ no. 36, pp. 56–65, 2014. [37] P. Baldwin and D. Price. (2017) Debategraph. [Online]. Available: [11] B. J. Robertson, “Organization at the leading edge: Introducing ho- http://debategraph.org/ lacracy,” Tech. Rep., 2007. [38] Loomio Cooperative Limited. (2016) Loomio - better decisions together. [12] Koumbit. (2017) About us. [Online]. Available: https://www.koumbit. [Online]. Available: https://www.loomio.org/ org/en/about [39] Mountain Goat Software. (2017) Planningpoker.com - estimates [13] Loomio Cooperative Limited. (2016) The loomio co-op handbook. made easy. sprints made simple. [Online]. Available: https://www. [Online]. Available: https://www.loomio.coop/ planningpoker.com/ [14] Zappos Insights. (2017) Holacracy - flattening the organization [40] Reddit. (2017) Reddit. [Online]. Available: https://about.reddit.com/ structure and busting bureaucracy. [Online]. Available: https://www. [41] Civilized Discourse Construction Kit. (2017) What is discourse? zapposinsights.com/about/holacracy - discourse - civilized discussion. [Online]. Available: http://www. [15] A. Doyle. (2016) Management and organization discourse.org/about at medium. [Online]. Available: https://blog.medium.com/ [42] holaSpirit SAS. (2017) holaspirit - application for holacracy and teal management-and-organization-at-medium-2228cc9d93e9 organizations. [Online]. Available: https://www.holaspirit.com/ [16] Drupal. (2016) Drupal code of conduct. [Online]. Available: https: [43] J. Janssen, M. De Cock, and D. Vermeir, “Fuzzy argumentation net- //www.drupal.org/dcoc works,” Vrije Universiteit Brussel, Tech. Rep., January 2008. [17] Seeds for Change, A Consensus Handbook. Footprint Workers’ Co-op, [44] C. Edwards, “Agreeing to fail,” Engineering & Technology, vol. 4, 2013. no. 7, pp. 74–75, 2009. [Online]. Available: https://doi.org/10.1049/et. [18] D. G. Messerschmitt, C. Szyperski et al., “Software ecosystem: under- 2009.0716 standing an indispensable technology and industry,” MIT Press Books, [45] B. Combemale, J. Deantoni, B. Baudry, R. B. France, J.-M. Jézéquel, vol. 1, 2005. and J. Gray, “Globalizing modeling languages,” Computer, pp. 10–13, [19] D. Zwieback, Beyond Blame, Learning from successes and failure. 2014. O’Reilly Media, 2015. [46] D. P. Twitchell, M. Adkins, J. F. Nunamaker, and J. K. Burgoon, “Using [20] A. Cornell, Oppose and Propose! AK Press, 2011. speech act theory to model conversations for automated classification [21] D. Vannucci and R. Singer, Come Hell or High Water. AK Press, 2010. and retrieval,” in Proceedings of the International Working Conference [22] R. Cialdini, “The science of persuasion,” Scientific American, vol. 284, Language Action Perspective Communication Modelling (LAP 2004), pp. 76–81, 2001. 2004, pp. 121–130. [23] F. J. Cabrerizo, F. Chiclana, M. R. Uren̋a, and E. Herrera-Viedma, [47] S. Joty and E. Hoque, “Speech act modeling of written asynchronous “Challenges and open questions in soft consensus models,” in 2013 Joint conversations with task-specific embeddings and conditional structured IFSA World Congress and NAFIPS Annual Meeting (IFSA/NAFIPS), models,” in Proceedings of the 54th Annual Meeting of the Association June 2013, pp. 944–949. for Computational Linguistics, ACL, 2016, pp. 7–12. [24] N. B. Moe, A. Aurum, and T. Dybå, “Challenges of shared decision- [48] R. P. Fawcett and B. L. Davies, Monologue as a turn in dialogue: making: A multiple case study of agile software development,” Informa- Towards an integration of exchange structure and rhetorical structure tion and Software Technology, vol. 54, no. 8, pp. 853–865, 2012, special theory, 1992, pp. 151–166. Issue: Voice of the Editorial Board. [49] E. Black, P. McBurney, and S. Zschaler, Towards Agent Dialogue as a [25] C. Zannier, M. Chiasson, and F. Maurer, “A model of design decision Tool for Capturing Software Design Discussions, 2014, pp. 95–110. making based on empirical results of interviews with software design- [50] A. S. Mashiyat, M. Famelis, R. Salay, and M. Chechik, “Using devel- ers,” Information and Software Technology, vol. 49, no. 6, pp. 637–653, oper conversations to resolve uncertainty in software development: A 2007. position paper,” in Proceedings of the 4th International Workshop on [26] M. Van Mechelen, M. Gielen, V. vanden Abeele, A. Laenen, and Recommendation Systems for Software Engineering, 2014, pp. 1–5. B. Zaman, “Exploring challenging group dynamics in participatory [51] S. Ghanavati, D. Amyot, and L. Peyton, “Towards a framework for design with children,” in Proceedings of the 2014 Conference tracking legal compliance in healthcare,” in Advanced Information on Interaction Design and Children, ser. IDC ’14. New York, Systems Engineering. Springer, 2007, pp. 218–232. NY, USA: ACM, 2014, pp. 269–272. [Online]. Available: http: [52] S. Kokaly, R. Salay, V. Cassano, T. Maibaum, and M. Chechik, “A model //doi.acm.org/10.1145/2593968.2610469 management approach for assurance case reuse due to system evolution,” [27] A. Kalyanasundaram, W. Wei, K. M. Carley, and J. D. Herbsleb, in Proceedings of the ACM/IEEE 19th International Conference on “An agent-based model of edit wars in wikipedia: How and when Model Driven Engineering Languages and Systems. ACM, 2016, pp. is consensus reached,” in Proceedings of the 2015 Winter Simulation 196–206. [53] S. A. Busari and E. Letier, “Radar: A lightweight tool for requirements [58] D. Atewologun, R. Sealy, and S. Vinnicombe, “Revealing intersectional and architecture decision analysis,” in Proceedings of the 39th dynamics in organizations: Introducing ’intersectional identity work’,” International Conference on Software Engineering, ser. ICSE ’17. Gender, Work & Organization, vol. 23, no. 3, pp. 223–247, 2016. Piscataway, NJ, USA: IEEE Press, 2017, pp. 552–562. [Online]. [59] S. Easterbrook, J. Singer, M.-A. Storey, and D. Damian, “Selecting em- Available: https://doi.org/10.1109/ICSE.2017.57 pirical methods for software engineering research,” Guide to advanced [54] G. Brunet, M. Chechik, S. Easterbrook, S. Nejati, N. Niu, and M. Sa- empirical software engineering, pp. 285–311, 2008. betzadeh, “A manifesto for model merging,” in Proceedings of the 2006 [60] C. Calhoun, Critical social theory: Culture, history, and the challenge international workshop on Global integrated model management. ACM, of difference. Wiley-Blackwell, 1995. 2006, pp. 5–12. [61] T. C. Lethbridge, S. E. Sim, and J. Singer, “Studying software [55] P. Brosch, G. Kappel, P. Langer, M. Seidl, K. Wieland, and M. Wimmer, engineers: Data collection techniques for software field studies,” “An introduction to model versioning,” in Proceedings of the 12th Empirical Software Engineering, vol. 10, no. 3, pp. 311–341, 2005. international conference on Formal Methods for the Design of Computer, [Online]. Available: http://dx.doi.org/10.1007/s10664-005-1290-x Communication, and Software Systems: formal methods for model- [62] P. S. M. Dos Santos, G. H. Travassos, and M. V. Zelkowitz, “Action driven engineering. Springer-Verlag, 2012, pp. 336–398. research can swing the balance in experimental software engineering,” [56] P. Kruchten, R. L. Nord, and I. Ozkaya, “Technical debt: From metaphor Advances in computers, vol. 83, pp. 205–276, 2011. to theory and practice,” Ieee software, vol. 29, no. 6, pp. 18–21, 2012. [63] H. In and S. Roy, “Issues of visualized conflict resolution,” in Proceed- [57] K. W. Buffinton, K. W. Jablokow, and K. A. Martin, “Project team dy- ings Fifth IEEE International Symposium on Requirements Engineering, namics and cognitive style,” Engineering Management Journal, vol. 14, 2001, pp. 314–315. no. 3, pp. 25–33, 2002.