Software product roadmaps are practical tools that provide direction for product development. Software product roadmapping combines the reasoning why something is done with what should be done, often in the form of items to be delivered when constructing a software product. Successful roadmapping activities require collaboration from multiple stakeholder groups, such as business, development and management. However, aligning company goals, business strategy and development eforts is far from trivial. To this end, we conducted an action research study investigating how information exchange should be supported in software product roadmapping. As our results, we contribute the open-sourced and provide insights on how information exchange should be supported in software product roadmapping. Roadmapper supports information exchange in software product roadmapping by allowing diferent parties to clarify their views and making them understandable to other stakeholders, facilitating the discussion when they meet. Thus, Roadmapper visualises a common situational picture of software product development and acts as a group memory - helping to remember what the other stakeholders think about the matter.
Roadmaps are practical tools for strategic planning as they facilitate communication between stakeholder groups, bringing together viewpoints from diferent parts of an organisation [1]. A roadmap refers to a plan that presents decision alternatives over time [2], to which the term roadmapping refers to the roadmap creation process [3]. Software product roadmapping combines the reasoning behind product development decisions, i.e. why something is done and what should be done, usually in the form of items to be delivered [4]. Clear strategy and goals, derived from company vision, play a central role in aligning software product roadmapping – having all involved stakeholders committed around the same plan [5]. However, practical business strategy and product development alignment are far from trivial [6, 7].
nEvelop-O
The rest of the paper is organised as follows. In Section 2, we provide the theoretical background of our paper and explore related scientific research. Next, we describe our research process in Section 3. The findings of this paper are presented in Section 4 and discussed in Section
5. Section 5 also presents avenues for further research. business and requirements engineering [17]. UnderstandFinally, the conclusions are given in Section 6. ing value creation from a customer perspective [17, 7] is vital for roadmapping activities, as fragmented customer knowledge is one of the main issues with software 2. Background and related work product roadmapping [7].
Roadmaps present decision alternatives over time [2], and roadmapping as an activity refers to the roadmap development process [3]. Software features are ”prominent or distinctive user-visible aspect, quality, or characteristic of a software system or systems” [10]. However, various definitions for features are available, and a universally agreed-upon definition has yet to emerge [ 11].
Several grey literature reviews – based on the analysis of articles, blogs, white papers, and books – have been conducted around software product roadmapping in recent years (see, e.g. [4, 12, 13, 5]). Software product roadmapping has been found to combine the reasoning behind product development decisions, i.e. why something is done and what should be done, usually in the form of items to be delivered [4]. Clear strategy and goals, derived from company vision, play a central role in aligning software product roadmapping – having all involved stakeholders committed around the same plan [5]. Traditional roadmapping approaches face several issues: the lack of shared and well-communicated product vision, not reviewing and updating roadmaps, failure to integrate client feedback channels into roadmapping, using the wrong tools, unrealistic expectations, lack of criteria for roadmap item prioritisation, and creating a single roadmap for all stakeholders [12]. Overly detailed, feature-driven roadmaps can be prone to failure due to the inability to react to changes in uncertain business conditions [12].
Software product roadmaps can be divided into four categories: feature-driven, goal-oriented, outcomedriven, and theme-based roadmaps [13]. Software product roadmapping as a process can be divided into five phases: capturing, analysing, and prioritising features, roadmap validation and agreement, and change management – of which prioritisation and capturing features are the most critical ones [14]. Previous research has shown that at least three perspectives should be present in feature prioritisation activities: development, finances, and clients [15, 16]. Other studies continue on similar lines, promoting customer and partner representatives, marketing, product management and development as key stakeholders in software product roadmapping [ 14].
Roadmaps can be efective tools for strategic planning by facilitating communication between stakeholders and bringing together viewpoints from diferent parts of organisations [1]. Increasing the visibility of software planning activities within companies is essential in linking
temporary software product companies must be able to deliver value to both themselves and their customers. Value-based software engineering [ 18, 19] (VBSE) is an approach that promotes thinking regarding value when making decisions in the context of engineering software. Feature value estimation is a multi-dimensional problem [20, 19] consisting of several dimensions: customer value, market competitiveness, economic value/profitability, cost eficiency, technology & architecture and company strategy [19]. However, best practices for value estimation are still missing.
One of the more interesting related research works is tied to the VALUE framework [21] and the VALUE tool [22, 23] – developed to assist companies in adopting value-based decision-making. The authors have observed improved stakeholder decision-making with the tool-assisted representation of stakeholder value propositions and facilitated meetings. Another example is the ReleasePlanner tool [24] which formally evaluates features by distilling their value from multiple criteria to produce optimal release plans. Other tools, such the Software Product Management Workbench [25], focus more on the product management aspects of roadmapping.
A study [26] was performed among more than 100 (Belgian) product builders, recognising six techniques to estimate and prioritise values of features: • Clustering in importance groups: approaches that divide the requirements into a small number of diferent importance groups • Consensus-based approaches: approaches specifically geared towards reaching consensus among a group of stakeholders. • Multi-criteria ranking: approaches that automatically rank the requirements, based on the value of multiple relevant criteria and a specific formula that combines these values into a single value. • Pairwise comparison: approaches that rely on mutually comparing all requirements, and identifying for each comparison the most valuable requirement. • Voting systems: approaches that involve different stakeholders and ask each one of them to express their preference in some way or another. Pre-study Interviews (N=12) and workshops (N=6)
Implementation Vincit Oyj building the first version of Roadmapper Action research cycles (N=3) Diagnosis Decision making for the cycle
Development of Roadmapper
Action taking and planning Testing of Roadmapper and scenario development
Evaluating Scenario execution and observation, feedback collection
Action group Bi-weekly meetings
Specification of learnings Analysis of scenario execution and feedback
• Financial approaches: approaches based on financial measures.
Furthermore, the authors [26] analysed these tech- DOM5 research project between September 2021 and niques in two dimensions: the number of requirements May 2022. The research project’s high-level goal was and uncertainty of value estimation – concluding that to develop new visualisations that combine data from none of the techniques works well in cases where there multiple sources in modern DevOps development [31] are many requirements and the uncertainty of value esti- and create visualisations related to the health of software mation is high. Development efort and costs are also one processes. This study is based on the results from an earof the more traditional ways of estimating the monetary lier study [30] from that project, conducted between 2019 impact of development and are notoriously challenging and 2020, to better understand stakeholders’ information to predict. Prior research has found that efort estima- and visualisation needs in agile software development. tions based on the subjective assessment of experts are The previous study included interviews and workshops the most prevalent efort assessment techniques in agile to identify the information needs of software practitionsoftware development [ 27]. ers. One of the main findings of the previous study was that unsystematic management of client feedback and feature requests resulted in several information needs 3. Methodology from multiple stakeholder perspectives. Thus, this study investigated further how input from multiple stakeholder This study aimed to support the software roadmap- perspectives can be combined collaboratively to improve ping process from the viewpoint of diferent stakeholder decision-making related to software product roadmapgroups. This goal was refined into the following research ping. question: How should information exchange be supported The design work of Roadmapper was initially based in software product roadmapping? We conducted an ac- on the findings from the previous study, where a docution research (AR) study [8, 9, 28, 29] in collaboration ment was composed to outline the goal of Roadmapper with Vincit Oyj4, or Vincit for short from here on, to and its initial features. The aim was to create a tool that answer our research question. Three iterative AR cycles visualises how software development eforts related to were conducted during this study (Figure 1). a higher-level plan – a goal, product vision or story, or other suitable aims that drive development in a specific direction. Collecting, estimating, and ordering development items was expected to increase visibility on the 4https://www.vincit.com/ 5https://itea3.org/project/visdom.html underlying business value expectations and workload es- ing facilitators of what should be done. The action team timations while deepening the understanding of current discussed each scenario’s general aim, after which Vincit feature evaluation criteria. Tool support was presumed produced a scenario that would represent as close as to provide visibility and traceability to software develop- possible to a typical roadmapping session for the comment decisions, benefiting stakeholder groups such as pany – correspondence to real-world usage determined management, sales and development as roadmaps can be by Vincit, within reasonable limitations. The resulting a base for communicating development plans and facili- scenario was then commented on by the researchers and tating a shared understanding of development goals and revised as needed. priorities. Three expert reviewers from Vincit represented the
Vincit agreed to build the necessary software to begin key stakeholders described in the scenarios: a PO, two studies with stakeholder information needs in software ADs, and one developer. The number of expert participroduct roadmapping. Vincit is a multi-national small pants was determined by the minimum number of roles and medium-sized enterprise (SME) established in Fin- needed to carry out the scenarios. Each of these roles land that specialises in software development, service had specific tasks: PO for leadership and product experdesign, consulting and software products. At the time of tise, ADs to evaluate the business value of features and the study, Vincit had around 470 employees globally. represent their client’s needs, and a developer with tech
The action research team consisted of four researchers nical knowledge of the product. For practical reasons, and one assistant from two universities, and of Vincit’s the selected developers were the ones that had developed employees – one manager responsible for the company’s the tool and could reliably answer technical questions research activities related to the tool, one user interface the expert reviewers might have. The reasoning for these (UI) designer, a senior developer, and three junior de- specific roles is further explained in Section 4.1. Vincit velopers that alternated participating in the action team selected the expert reviewers based on their experience meetings. Vincit’s participants were selected by the man- and availability for the sessions. ager responsible for their research activities based on the The scenario descriptions were sent to the three exavailable resources and the team’s skills, interests and pert participants as attachments to the session invitations. availability. The action team held bi-weekly remote meet- The participants were also given customised descriptions ings to synchronise progress, which researchers recorded of specific needs for their dedicated clients in the siminto a research diary shared between the action team in ulation and a more concrete mission to achieve during Google Drive6. The research diary included topics of the session, such as “In the upcoming roadmap planning discussion, possible research avenues, decisions made meeting, your job is to make sure that your customers get and recordings of dates and participants. The action the features they need on the roadmap.” The features team aimed to develop a tool to assist software product to be discussed in the expert evaluation sessions were roadmapping by providing information from multiple the Roadmapper tool’s backlog tickets, imported by the stakeholder perspectives. software developers from Trello 7. An example task description can be seen as “Auto-accept user invitations on
Roadmapper was developed to include the necessary The Roadmapper tool was developed during three itera- views and features to support software product roadmaptive AR cycles by Vincit while observed and evaluated ping tasks before the evaluation sessions. The tool was by the research partners. Each cycle followed similar further developed during the AR cycles based on the feedsteps: 1) issues were diagnosed to be solved during each back from previous cycles. For each evaluation session, development cycle, 2) a simulated roadmapping session the Roadmapper was pre-filled with imported Trello tickwas planned and conducted with three expert reviewers, ets from the development backlog, called tasks within the 3) researchers analysed the results to evaluate whether Roadmapper, related to the scenario description to simuthe diagnosed issues were solved, and 4) researchers spec- late ongoing development work in a product. Each task ified the lessons learned, which were validated by the was pre-evaluated for the sessions based on their value expert reviewers. for specific clients and overall implementation complex
Several documents were composed in collaboration ity. The participants were each given tasks to be added with company representatives before any sessions were based on the scenario description. Then, ADs were reconducted: facilitator guidelines, scenario descriptions sponsible for evaluating task value for their clients, and for each session, and invitation letters for the partici- developers were responsible for evaluating task implepants. The facilitator guideline included instructions on mentation complexity. When these initial steps were conducting the sessions from beginning to end, remind- complete, the participants were free to organise the soft
ware product roadmap using the tool as they pleased to results cover three main areas of interest: specifying the complete their mission in the scenario. necessary information content and representation for
The expert evaluation sessions were conducted in a hy- supporting information exchange, how value and combrid setting. The study participants, company observers, plexity estimation was perceived by the participants, and and researchers from Tampere University were located answering the question of what to do next based on this in a meeting room within the company premises. Re- information via collaborative roadmap planning. searchers from the University of Oulu participated remotely due to the travelling restrictions related to the 4.1. Information contents of Roadmapper COVID-19 pandemic at the time. The sessions were recorded via Microsoft Teams 8 with informed consent, Roadmapper was designed to facilitate information exand privacy procedures were explained to the partici- change in software product roadmapping with three main pants. The facilitator described the session for the partic- types of stakeholders in mind: product owners that proipants, led the session’s activities, asked clarifying ques- vide the direction and vision for software product detions, noted the time used, and kept the session moving velopment, account directors that know their clients and if the participants needed help with what to do. Other their needs within the product’s context, and developers participants observed the sessions and took notes. The who provide technical expertise on implementing the PO was asked to share their screen in each session and requested changes to said product. Roadmapper was lead discussions with ADs and software developers to intended to aid in discussion sessions led by the PO, comcreate a balanced software roadmap. mitting the relevant stakeholders to the same action plan
After each session, one researcher watched the related concerning the product. Thus, the essential point is that recording and created an annotated file by marking the one should not think of our results only from a technical time when something interesting happened or was dis- roadmapping perspective but as an attempt to support cussed and wrote a description of what was discussed information exchange in general – where supporting the without any further analysis. An example of an anno- needs manifests as a roadmapping tool. tation can be seen in “57:22 - Participant C noted that Roadmapper is divided into five main views. The first development always featured surprises and that the total default view is the dashboard that visualises the current amount of time would not be completely known. They roadmap’s value output of the current plan versus a nunoted that a value indicating development speed (in the merically optimal plan. The dashboard also includes all same unit that work estimates use) would be useful.” After tasks that still need to be rated. The second main view completing the annotated file, the notes were grouped is the tasks view, providing an overview and details of by their overarching theme, such as the numerical scale all the tasks within the project. The third view, clients, of feature evaluation. Then, the notes under each theme represents a list of clients interested in the product being were re-read and condensed into a description that con- developed. The weights of each client can be altered to afveyed the essence of each theme. fect the optimal roadmap. The fourth view, the team view,
The themes, their meaning and proposed actions to provides a way to manage the persons included within address deficiencies were reported each iteration in an the selected project, selecting their roles and granting analysis document shared within the action team. In ad- them access to specific clients. The final view, the plan dition to research-related topics, any encountered bugs in tab, is the most important as it features the functionality the tool were reported in the analysis document. The re- needed to create milestones and visualise their current sults were then further condensed as Powerpoint9 slides composition (Figure 2). for presentation to the expert participants for validation. Conceptually, Roadmapper deals with development Based on the results, the researchers made recommen- items in a ticket format imported from external project dations for actions for the next cycle, which were then management tools such as Trello, Jira10 and Gitlab11. The discussed with the company participants when planning tickets within those systems, with abstraction levels corthe next cycle. The action team then selected the issues responding to each company’s practices, are called tasks to be addressed in the next cycle. within the Roadmapper. Each task includes ratings for value and complexity. Account directors are responsible for rating task value for each of their clients: how valu4. Results able that item would be from the client’s perspective. On the other hand, developers evaluated complexity, estimatWe report our results and lessons learned from the view- ing the total challenge and efort needed to implement point of how and why the Roadmapper tool was con- said task to the product. structed as it was based on the three AR cycles. Our
Besides numeric metrics, each task was assigned de- are collections of tasks. The milestones are read from left pendencies and synergies concerning other tasks (Figure to right, the leftmost representing the next development 3). Dependencies dictate whether a task must be com- cycle. Each milestone provides average and total scores pleted before other tasks from a technical point of view or for values along a client share score, denoting how well the other way around. The synergy between tasks repre- the proposed milestones follow the current client weights sents an alternative, optional way of evaluating task value set by the PO. Each client score can be set to denote their – developing synergistic tasks simultaneously within a importance to the company using Roadmapper, assumspecific milestone would increase the roadmap’s overall ing that a software product is being developed to be used value creation concerning the efort spent (complexity by multiple client organisations. Milestone planning in within Roadmapper). Roadmapper provides discussion support for the collabo
Projects represent the most high-level concept within rative identification of valuable tasks, ordering them to Roadmapper, each project including one roadmap. Indi- deliver value. vidual roadmaps consist of milestones (Figure 4) – which The expert participants all agreed that the tool, to be helpful, should focus on simplicity to allow for flexibility over rigid practices. This guiding principle was followed during the tool’s construction and provided critical reasoning on why it ended up as it did. Concrete examples of this principle can be found in how value and complexity are estimated and how the tool always reflects the current situation and is updated as needed. The core idea of the tool was not to represent everything perfectly but to present the necessary information in an understandable format to support collaboration and information exchange.
engineering – and an essential aspect of the tool since it relies on providing numerical task values. The three AR cycles showed that two main design aspects received the most feedback from the participants: the numerical evaluation scale of value and complexity, and interpreting criticality of tasks – evaluating tasks between the opposite ends of the spectrum in importance: irrelevant or business critical for specific clients.
We found that a five-step scale proved the most efective in rating tasks when accompanied by textual descriptions of the values to guide the scale’s usage between stakeholders. The following scale was used in the third iteration: 1 for Not relevant, 4 for A bit important, 9 for Somewhat important, 16 for Important, and 25 for Business critical. Three evaluation scales were tested in total – a linear scale from one to ten, a linear scale from one to five, and the third non-linear five-step scale from one-to-25 – and ended up with the last scale as it best corresponded to participant needs and produced noticeable value diferences between tasks.
The business criticality of tasks was often referred to as the most important aspect from a customer viewpoint. Whether or not delivering the task would have ramifications with doing business with a specific client. Several discussions were had about evaluating business criticality separately or indicating it with flags, but the consensus remained to keep the tool simple to allow for flexibility. The caveat of not including separate indicators for business criticality is that one task may be critical for one client and irrelevant for others, implying that the criticality of tasks may only sometimes be related to the overall business value of tasks – another example of why supporting discussions in product roadmapping are essential for companies. Another issue uncovered was rating tasks when ADs were unsure of the task’s value – situations such as when a task might have value to the company, but further discussions were required to determine the actual value. Resolving this issue was left as future work: determining whether the task is good enough to decide on and how to indicate it. Lastly, free-form comments within tasks were considered helpful in conveying information such as layouts for planned features and implementation deadlines – dates that determined when the feature had to be complete and delivered to receive any value, such as before an event.
For future work, a technical debt score was requested to dictate how the roadmap should be balanced towards the company’s own needs, which could help communicate long-term commitments and maintainability towards clients. Developers would evaluate technical debt from a technical point of view and display it as a core metric along with task value and complexity.
Milestone planning lies in the heart of Roadmapper, where the participants answered the question of what to do next based on this information via collaborative to make a pretty far-reaching ready-made roadmap planning. This view combines the knowledge of guess that could be reviewed with the aceach perspective into action, where the inputs of every- counts.” (PO) one involved are resolved to a common cause. Following the principle of simplicity and flexibility, we found that For future reference, the participants argued that inieach task must contain the following information for tial software roadmaps could also be created automatquick milestone planning: how complex each task was to ically if task dependencies, the value and complexity complete, how much value the task would create when of tasks, client weights and team capacity were known. completed, and which tasks were essential to clients – Furthermore, task synergy estimation was considered confirming the initial presumptions on the tool’s design. helpful and highlighted the collaborative nature of plan
The milestones in Roadmapper were purposefully left ning when deciding which items should be completed in more abstract than they could have been, once again which development cycle. echoing the principle of simplicity and flexibility. Even The openness of information was considered essential though the participants intuitively thought of milestones by the participants. Specific views, such as the milestone as scrum sprints, the abstraction level and purpose of planning view, were initially restricted to being accessible creating milestones varied. The participants argued that only to the PO. However, all the participants agreed that some tasks, such as hot-fixes, were out of scope for this open information sharing within the tool was preferred kind of tool and placed these kinds of tasks directly to over restricting access to information. The participants strategic development – meaning in practice that they highlighted the tool’s conversational nature, where some were added to the current or next development sprint roadmapping insights only surface in joint sessions when as priority tasks and purposefully not included in the looking at the same screen – showing the screen to other roadmap. An essential mental distinction is that the roles leads to making specific observations earlier. In roadmap was not considered a concrete sprint plan either, practice, the participants agreed that it would benefit supporting the findings of the conducted pre-study. Task ADs and developers to access all parts of the tool, with complexity is abstracted away from actual working hours, read-only access at the very least. even though the tool provides a sanity-check mechanism The role of ADs was to support the PO while constructfor comparing the complexity of milestones to estima- ing roadmaps and provide client-related knowledge for tions of real working time. Nevertheless, this feature the PO by rating task values from the viewpoint of their is intended to judge the appropriate sizes of milestones clients. A concrete visualisation was suggested as a list against each other. of tasks sorted by highest value and lowest efort con
The participants created software roadmaps with what cerning the specific client to help communicate their was titled “the greedy algorithm”, first doing tasks that most pressing needs to the PO. Furthermore, understandprovided the most value with the least complexity. The ing the status of each customer using the product and PO determined the milestones within a roadmap using providing communication assistance towards the clients the following process. First, team velocity was needed was considered a priority for further development action to determine the correct size of a milestone. Then, a bal- by the ADs. The ADs were interested in knowing how anced composition of tasks was collected within said mile- each client was doing regarding overall value output and stone based on client weights or other explicitly stated whether their needs were being served. needs – repeated until as many milestones as needed were created. The “greedy algorithm” did, however, face issues that were expected by the action team, further verifying the initial presumptions: the dependencies of tasks must be known before they can be meaningfully arranged. “What’s coming [features to be delivered] and a rough estimate of when it’s coming.
After all, it could show the tasks that generate a lot of value for the customer in question on a timeline and what comes from there at any time.” (AD1) “As PO, I think that if I have an understanding from the technical experts that there are dependencies here or that these should be done at the same time. If I have an understanding from the account directors that how much this particular ticket serves this customer, with what kind of weightings and what is the order of priority of the tickets from that account’s point of view, then I would believe that as a PO I would be able
Finally, visually and conceptually explicating both technical and value-related information was considered to be the most valuable asset of Roadmapper, to which the PO and ADs provided the following shared testimony in agreement: “[the Roadmapper tool] would help. I have done roadmapping with diferent tools.
When [using other tools, you’re] carrying similar types of information with other ways, or by asking, or in conversations, but here we get technical expertise information in a formal way, it is shown and it can be utilised when planning. And in the same way numerical information can be obtained from accounts [account directors], of course it helps. Information on the basis of which one does one’s own work seems pretty good based on this.” (PO)
tween stakeholder groups in software product roadmapping. To this end, we contribute an open-sourced software product roadmapping tool called Roadmapper and specify lessons learned during its construction with Vincit. dependencies of tasks. ADs provided abstracted value estimations on how beneficial the features would be from their customers’ perspectives. Roadmapper supports information exchange by providing the means to collect and present information the most relevant for each stakeholder group, allowing them to discuss it and form a Roadmap that provides a steady output of value while considering necessities such as technical dependencies.
Roadmapper helps address the most critical and problematic phases of product roadmapping: prioritising features [14]. Roadmapper does this by fostering communication between stakeholder groups and providing visual aids to obtain a joint agreement on the product roadmap.
Furthermore, Roadmapper helps address several other issues reported in contemporary roadmapping, primarily: 5.1. RQ: How should information exchange be supported in software product roadmapping? • The need for shared and well-communicated product vision. [5, 12] Roadmapper-facilitated discussions are aimed at providing a joint agreement on the roadmap, where the PO is responsible for following and communicating the company vision. • The need for integrating client feedback channels into roadmapping. [12] When ADs know their clients’ needs, their primary responsibility in Roadmapper-facilitated discussions is to ensure their clients receive value and remain satisfied with the current product roadmap. • The need for criteria in roadmap item prioritisation. [12] Roadmapper provides explicit means and metrics to prioritise features: customer value, task complexity, technical dependencies, and synergy between features. Each participant is able to see the criteria used for decisionmaking, increasing the transparency of product development decisions.
Software product roadmapping allows stakeholders to commit to a shared product development plan [5] by combining perspectives from diferent parts of an organisation [1]. However, aligning business strategy and product development remains challenging in practice [6, 7]. To this end, we created a solution primarily focused on supporting the discussion between stakeholder groups to align their perspectives for a shared product development plan. Besides Roadmapper itself, the main results of this study are the lessons learned on how tool support can help foster collaboration in software product roadmapping. We highlighted three main areas of interest in our study: the information content and representation for supporting information exchange, how value and Guided by the principle of simplicity, Roadmapper complexity estimation was perceived and done by the provides an open-ended and flexible way of evaluating participants, and how milestone planning tied these as- feature value in a software product roadmapping conpects together into actions by answering the question of text. In comparison to other tools, unnecessary comwhat to do next based on this information via collaborative plexity was found to be one of the VALUE tool’s main roadmap planning. weaknesses [22]. Tools such as ReleasePlanner [24] and
We successfully supported information exchange in Software Product Management Workbench [ 25] also rely software product roadmapping with role-specific infor- on a more formal development planning. However, in mation representation. According to previous research, this study the participants all preferred the free-form development, finances, and clients’ perspectives should way of roadmapping that focused on ordering feature be present in feature prioritisation activities [15, 16]. delivery based on value output and implementation comRoadmapper follows a similar cast of stakeholders by plexity, while indicating implementation synergies and including a technical perspective from development, a dependencies. customer value perspective conveyed by ADs, and a prod- Based on our findings and the lessons learned while uct leadership perspective from a PO. Both developers constructing Roadmapper, we conclude that abstractions and ADs were responsible for rating tasks according to of feature value for clients and implementation complextheir speciality: developers provided an abstraction of ity, coupled with task dependencies and synergies, form task complexity, implementation synergies and technical a base on grouping and ordering tasks for a software product roadmap. Increasing the visibility of presump- oped as it was. The findings of each cycle were reported tions and expectations of value and complexity promotes back to the participants for verification, and they were decision-making transparency for all participants and given a chance to comment on them – grounding them aids in achieving a joint consensus between stakeholder through expert review and validation. However, we congroups when using the tool to support product develop- sider it part of future work to see whether the conclusions ment discussions. Finally, the following lines of research presented in this paper can be applied to projects using provide opportunities for further research. diferent development methods from a general software
The first avenue for future research is value estimation engineering viewpoint. in software engineering. Estimating value for features Lastly, two researchers participated actively in the and other development items, such as paying back tech- research cycles, and three other researchers participated nical debt, is often challenging. Feature synergy was one in the validation of the results as outsider participants of the core ways of estimating the value of features in to reduce the risk of bias due to being an active part of this study. Future research should investigate how the the research. The company participants conducted the value of diferent development items can be estimated actual development. and communicated transparently.
Pre-emptive estimation of technical debt represents the second direction for future research. Implementing 6. Conclusions software features often incurs technical debt. Future research should investigate how decision alternatives in product roadmapping afect the upcoming technical debt of the product.
Lastly, supporting customer-directed communication ofers a third avenue for future research. Roadmappingrelated information is often easier to communicate to internal stakeholders than external stakeholders. Future research should investigate how customer-directed communication can be supported when making product development decisions.
Roadmaps are valuable tools that represent decision alternatives over time, combining the why and what should be done in software product development. However, aligning business strategy and product development in practice remains challenging. To address this challenge, we conducted an AR study on how information exchange should be supported in software product roadmapping.
To this end, we contribute an open-sourced software product roadmapping tool titled Roadmapper. In addition, we specify lessons learned in supporting the exchange of information. We emphasise that the tool shows promise in supporting information exchange in software product 5.2. Validity and limitations roadmapping by explicating tacit knowledge, promoting There are several limitations associated with this study. the visibility of estimations, transparency of decisionThe participants of this study and their number were making, and promoting the voice of each stakeholder decided by the hosting company, representing a selection group to be heard within those discussions – despite the of personnel suitable for expert reviews. However, the PO small sample of participants and the simulated nature of noted that they lacked experience in this specific kind of the evaluation sessions. work where a product was sold to multiple clients, despite The main takeaway from this study is that role-specific having strong experience otherwise. One potential flaw information representation was found to aid software in the simulation is that the developers were part of the product roadmapping by facilitating information exaction team and those that built the tool in reality. The change between stakeholder groups. Roadmapper suprole of developers in these kinds of discussions should ports information exchange by allowing diferent parties be studied further in future research. to clarify their views and making them understandable
The Roadmapper’s backlog items were used instead to other stakeholders, facilitating the discussion when of actual cases for confidentiality and practical reasons, they meet. Thus, Roadmapper visualises a common sitwhich led to simulated scenarios that may have afected uational picture of software product development and how the participants approached using the tool. The first acts as a group memory – helping to remember what iterations sufered from technical immaturity, potentially the other stakeholders think about the matter. The findleading to diferent results given more development time ings from this study related to providing the necessary each cycle. However, the discussions saturate towards information contents for software product roadmapping, the third session, giving the impression that the tool discussing value and complexity estimation of tasks, and fulfilled its purpose with its current features. outlining how communication in milestone planning can
We address validity, concerning the qualitative aspects be supported. Abstractions of feature value and impleof this study, through a detailed description of data [32] mentation complexity, coupled with task dependencies by providing supporting quotations for the presented and synergies, were found to form a base for software ifndings and reasoning how and why the tool was devel- product roadmaps. support for value-based decision-making: A usability assessment, in: 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), 2016, pp. 34–41. doi:10.1109/SEAA.
2016.44. [23] V. Freitas, M. Perkusich, E. Mendes, P. Rodríguez,
M. Oivo, Value-based decision-making using a web-based tool: A multiple case study, in: 2017 24th Asia-Pacific Software Engineering Conference (APSEC), 2017, pp. 279–288. doi:10.1109/APSEC.
2017.34. [24] G. Ruhe, M. Saliu, The art and science of software release planning, IEEE Software 22 (2005) 47–53.
doi:10.1109/MS.2005.164. [25] I. V. De Weerd, S. Brinkkemper, R. Nieuwenhuis,
J. Versendaal, L. Bijlsma, On the creation of a reference framework for software product management: Validation and tool support, in: 2006 International Workshop on Software Product Management (IWSPM’06 - RE’06 Workshop), 2006, pp. 3–12.
doi:10.1109/IWSPM.2006.6. [26] T. Tourwé, W. Codenie, N. Boucart, V. Blagojević, Demystifying release definition: From requirements prioritization to collaborative value quantification, in: Proceedings of the 15th International Working Conference on Requirements Engineering: Foundation for Software Quality, REFSQ ’09, Springer-Verlag, Berlin, Heidelberg, 2009, p. 37–44.
doi:10.1007/978-3-642-02050-6_4. [27] M. Usman, E. Mendes, J. Börstler, Efort estimation in agile software development: A survey on the state of the practice, in: Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, EASE ’15, Association for Computing Machinery, New York, NY, USA, 2015, pp. 1–10. doi:10.1145/2745802.2745813. [28] J. McKernan, Curriculum action research: A handbook of methods and resources for the reflective practitioner, Psychology Press, 1996. [29] R. L. Baskerville, A. T. Wood-Harper, A critical perspective on action research as a method for information systems research, Journal of information
Technology 11 (1996) 235–246. [30] H. Bomström, M. Kelanti, E. Annanperä,
K. Liukkunen, T. Kilamo, O. Sievi-Korte, K. Systä, Information needs and presentation in agile software development, Information and Software Technology (2023). doi:10.1016/j.infsof.2023.107265. [31] C. Ebert, G. Gallardo, J. Hernantes, N. Serrano, Devops, IEEE Software 33 (2016) 94–100. doi: 10.1109/
MS.2016.68. [32] J. W. Creswell, D. L. Miller, Determining validity in qualitative inquiry, Theory Into Practice 39 (2000) 124–130. doi:10.1207/s15430421tip3903\_2.