=Paper=
{{Paper
|id=Vol-1806/paper01
|storemode=property
|title=Towards a Systematic Approach to Integrate Usage and Decision Knowledge in Continuous Software Engineering
|pdfUrl=https://ceur-ws.org/Vol-1806/paper01.pdf
|volume=Vol-1806
|authors=Jan Ole Johanssen,Anja Kleebaum,Bernd Bruegge,Barbara Paech
|dblpUrl=https://dblp.org/rec/conf/se/JohanssenKBP17
}}
==Towards a Systematic Approach to Integrate Usage and Decision Knowledge in Continuous Software Engineering==
https://ceur-ws.org/Vol-1806/paper01.pdf
2nd Workshop on Continuous Software Engineering
Towards a Systematic Approach to Integrate Usage and
Decision Knowledge in Continuous Software Engineering
Jan Ole Johanssen1, Anja Kleebaum2, Bernd Bruegge3, Barbara Paech4
Abstract: Continuous software engineering (CSE) employs activities such as continuous integra-
tion and continuous delivery to support software evolution. Another aspect of software evolution is
knowledge management. There are two important knowledge types: usage knowledge derives from
explicit and implicit user feedback and helps to understand how users utilize software. Decision
knowledge encompasses decisions and their rationale on all aspects of the software lifecycle. Both
knowledge types represent important information sources for developers to improve the CSE activ-
ities and the software product. We envision an integration of usage and decision knowledge in the
CSE lifecycle. This extension consists of a monitoring and feedback component for user understand-
ing as well as a knowledge repository and dashboard component for knowledge visualization and
analysis. Usage and decision knowledge introduce challenges when integrating them in CSE. In this
paper, we present our vision and detail the challenges.
Keywords: Continuous Software Engineering, Usage Knowledge, Decision Knowledge
1 Introduction
Continuous software engineering (CSE) has been described by Bosch [Bo14] as a holistic
approach for software engineering that consists of several activities covering the complete
software lifecycle including continuous integration and continuous delivery.
Fitzgerald and Stol indicate the importance of monitoring any behavior during the run-time
of a system in CSE [FS15]. We refer to insights derived from users either through testing
or normal use as usage knowledge. Decisions made during the CSE activities represent
another type of knowledge: decision knowledge. Creating and managing decision knowl-
edge has already been explored by several researchers such as Dutoit et al. [Du06], while
they focused on design-time knowledge only. However, with its focus on evolution during
run-time, CSE introduces new challenges on managing usage and decision knowledge.
We have already worked on the integration of both usage and decision knowledge in non-
CSE environments: for instance, regarding the analysis of user behavior, we compared
monitored user interaction to use cases [Ro13]. Furthermore, regarding decision manage-
ment, we developed the decision documentation model as well as corresponding tool sup-
port and explored its application [HKR16].
1 Technische Universität München, Munich, Germany, johansse@in.tum.de
2 Universität Heidelberg, Heidelberg, Germany, anja.kleebaum@informatik.uni-heidelberg.de
3 Technische Universität München, Munich, Germany, bruegge@in.tum.de
4 Universität Heidelberg, Heidelberg, Germany, paech@informatik.uni-heidelberg.de
Copyright c 2017 for the individual papers by the papers’ authors. Copying permitted for private and academic
purposes. This volume is published and copyrighted by its editors.
7
2nd Workshop on Continuous Software Engineering
Up to now, usage and decision knowledge have not been explored within CSE. In our re-
search project CURES (”Continuous Usage- and Rationale-based Evolution Decision Sup-
port”), we plan to integrate both knowledge types in CSE. Beside their individual benefits,
we expect to leverage synergies in their combination: for instance, decisions might rely
on results derived from detected usage patterns. In addition, both face similar questions
regarding a consistent way of defining and storing the right granularity of information.
In the following, we present our vision of integrating usage and decision knowledge in
CSE. Thereafter, we outline integration challenges and describe our status and next steps.
2 Vision
As shown in Fig. 1, we envision an extension to CSE in which developers and users act
as the main stakeholders. As for the CSE infrastructure, we plan to employ Rugby, an ag-
ile process model for continuous delivery which builds upon event-based releases [Kr14].
Developers utilize feature branches to add product increments in form of code commits.
Feature branches are merged from and back to a master branch, which contains the final
software product. Each feature branch on its own can be released to users, allowing the
delivery of different proposals for one feature at the same time. Using a monitoring and
feedback component, developers can examine usage information that is mapped to indi-
vidual releases by applying approaches such as A/B testing for implicit usage information.
A knowledge repository continuously stores all information related to the development
and monitoring process. Furthermore, it maintains decisions related to feature branches.
Thereby, rationale can be accessed, visualized, and analyzed using a dashboard compo-
nent. For instance, the claimed solution to an issue could be compared to an alternative
solution based on the impact of a feature branch in the context of previous decisions. This
enables the developer to interact and reflect on collected usage and decision knowledge.
Developer User
uses release gives feedback
interacts, reflects monitors
creates
branch, usage Release in Operation
Dashboard commit,
release Monitoring and Feedback
analyses, visualizes
FB1 FB2 FB3
Knowledge Repository CSE Infrastructure
R1 R2 R3
F1 F2 Master Branch
P1 D1
P1 P2a P2b Feature Branch 1 D2
P2a
FB1 FB2 FB3 Feature Branch 2a
P2b
D1 D2
Feature Branch 2b
Fig. 1: Interaction between CSE Infrastructure, Knowledge Repository, and Dashboard: Feature F is
based on a Proposal P, Feedback FB is collected from a Release R and leads to a Decision D.
8
2nd Workshop on Continuous Software Engineering
For example, a feature F1 is developed based on a proposal P1 . After providing the release
R1 to users and analyzing feedback FB1 , the developer makes decision D1 and merges F1
into the master branch. In the same manner, different proposals P2a and P2b for a feature
F2 are evaluated. Feedback FB2 stops the work on P2a , while FB3 leads to decision D2 .
This CSE infrastructure extension forms a continuous knowledge activity and enables the
exploitation and documentation of knowledge acquired during CSE. It supports the evolu-
tion of knowledge and handles it similar to the evolution of code in a CSE infrastructure.
3 Challenges
In this section we describe the challenges of integrating usage and decision knowledge in
CSE and highlight references from the extension introduced in Fig. 1 in bold type.
There is a heterogeneous set of users. In CSE, the user role might be filled by the devel-
opers themselves, the customers who initiate the development of an application as part of
a contract, or the actual end users of an application. All of them have different intentions,
expectations, and approaches towards using an application which needs to be considered.
User feedback can either be explicit or implicit. Explicit feedback is actively provided
by the user, such as an app store review, while implicit feedback relates to anything users
reveal through their interaction, such as pressing a button. It has yet to be answered how
both feedback types can be incorporated within the CSE infrastructure: different strategies
for feedback elicitation need to be evaluated, such as in-application feedback requests for
new features or automatic usage data collection for each feature branch.
Enriching users’ explicit with implicit feedback in a monitoring and feedback component
enables the creation of a comprehensive user understanding. A main challenge is to derive
a user’s motivation and construct a user behavior model. Additional context information,
such as a user’s location or current activity, might be useful for creating such a model.
Further challenges arise in the synchronization of releases, feedback, and their granularity:
in a CSE infrastructure new increments are released frequently, while usage patterns
develop only over time and the difference between releases can be as little as a commit.
The decision documentation model is our starting point to integrate decision knowledge in
CSE since it allows for an intertwined documentation across agile development activities.
We want to address the following challenges towards a continuous rationale management.
Due to CSE’s focus on run-time, decision making is a high-frequency process and leads to
large amounts of data. Developers utilize different components, such as issue tracking and
version control systems. Knowledge is distributed across these components: for instance,
requirements are stored in the issue tracking system, whereas source code is stored in
the version control system. It is a challenge how distributed decision knowledge can be
systematically managed and synchronized during CSE as part of a knowledge repository.
9
2nd Workshop on Continuous Software Engineering
Regarding the decision knowledge repository further questions arise: (1) Which decisions
are worthwhile to be captured and managed? Should they be captured within the feature
branch or the master branch? (2) What is the right granularity of decisions and the related
rationale? (3) How to enable a seamless switch between coarse and fine-grained decision
knowledge? (4) What is the most suitable way to search for decision knowledge related
to a problem during software evolution? How should the knowledge be presented in the
dashboard? (5) How can a change impact analysis on decision knowledge be performed
efficiently and effectively?
4 Status and Next Steps
Currently, we work on an empirical study to investigate on CSE needs in practice. The goal
is to determine how developers deal with usage and decision knowledge to align the results
with our next steps. We develop a decision documentation editor for JIRA and Git. This
is our initial step to integrate the decision documentation model in the CSE infrastructure.
We plan on applying machine learning techniques to classify implicit user feedback.
5 Conclusion
Usage and decision knowledge has not been explored in CSE up to now. We described our
vision for a systematic approach to integrate both aspects in CSE. We identified challenges
such as the heterogeneity of users and differences in explicit and implicit feedback. An-
other challenge is the alignment of decision making processes with respect to the fast pace
of CSE activities such as continuous delivery, resulting in questions on decision capturing
and granularity. Finally, we outlined our status and next steps for the integration of usage
and decision knowledge in order to improve existing CSE infrastructures.
Acknowledgement
This work was supported by the DFG (German Research Foundation) under the Priority
Programme SPP1593: Design For Future – Managed Software Evolution.
References
[Bo14] Bosch, Jan: Continuous Software Engineering: An Introduction. Springer, 2014.
[Du06] Dutoit, Allen H; McCall, Raymond; Mistrı́k, Ivan; Paech, Barbara: Rationale Manage-
ment in Software Engineering: Concepts and Techniques. Springer, 2006.
[FS15] Fitzgerald, Brian; Stol, Klaas-Jan: Continuous software engineering: A roadmap and
agenda. Journal of Systems and Software, 2015.
10
2nd Workshop on Continuous Software Engineering
[HKR16] Hesse, Tom-Michael; Kuehlwein, Arthur; Roehm, Tobias: DecDoc: A Tool for Docu-
menting Design Decisions Collaboratively and Incrementally. In: Proceedings of the 1st
Int. Workshop on Decision Making in Software ARCHitecture. pp. 30–37, 2016.
[Kr14] Krusche, Stephan; Alperowitz, Lukas; Bruegge, Bernd; Wagner, Martin O.: Rugby: An
Agile Process Model Based on Continuous Delivery. In: Proceedings of the 1st Interna-
tional Workshop on Rapid Continuous Software Engineering. pp. 42–50, 2014.
[Ro13] Roehm, Tobias; Bruegge, Bernd; Hesse, Tom-Michael; Paech, Barbara: Towards Identi-
fication of Software Improvements and Specification Updates by Comparing Monitored
and Specified End-User Behavior. In: Proceedings of the 29th IEEE International Con-
ference on Software Maintenance. pp. 464–467, 2013.
11