Agile software development methodologies have been a focal point of industry and academia over the past two decades. Organizations are interested in extracting the value of agile methodologies and increasing their business success, however, it is evident that merely applying agile practices does not necessarily make organizations agile. We are building a context agnostic agile assessment model based on agile values and principles to enable contextually appropriate self-assessments, and support organizations' agile transformation endeavors. In this paper, we share the results of our expert interviews with the focus on two principle pillars: Human Centricity and Technical Excellence. Results of this study show that the proposed assessment questionnaire appropriately addresses highly relevant aspects of agility. It is found practically useful by the experts, yet the coverage of human centricity targeted questions can be improved.
As we approached the end of second decade mark of the publication of the Agile Manifesto [
Therefore we focus on the values and principles of the Agile Manifesto in building an agile assessment model (AAM). With this agile assessment model, we aim to support organizations nEvelop-O (R. Plösch) in their transformation journeys and help them identify how agility is lived within their units, irrespective of any agile framework. This model, in our opinion, has the potential to support organizations in conducting contextually appropriate (self)-assessments, judging their own capabilities with respect to their own targets, and identifying areas of improvement that are not pre-imposed as global optima by any of the agile frameworks.
During our previous research [
Although the industrial relevance of the problem is clear, there have been only a few attempts
for practically validating the existing models. Given the additional limitations of the existing
models we discussed in [
The main contribution of this research is the result of the expert interviews we conducted across diferent business units of Siemens towards the validation of the questions for Human Centricity and Technical Excellence principles. We select these two principles because if we can successfully design a robust question development mechanism to serve the needs of these two principles with two distinct stances, this will guide us for the remaining three principles, whose stances fall in between human centricity and technical excellence. The validation focuses on (1) the understandability and feasability of the questions for these principles and (2) the completeness of the questions with respect to the principles. This validation is necessary and valuable for our future research, as it allows us to adjust our self-defined questionnaire development guidelines, and to incorporate the feedback in the development of the questions for the remaining principles.
The structural components of the proposed agile assessment model have been discussed in our
previous paper [
Our model consists of two main structural components: Principles and Clusters. These structural
components correspond to the static aspect of the overall assessment system. Principles are the
abstract components that are constructing the areas under which agility should be evaluated.
After an extensive literature review, the principles are derived particularly from the model
proposed by Sidky et al. [7]. Clusters, on the other hand, represent one lower level of abstraction,
containing groups of certain practices of agile software development methodology. We are
reusing the principles and clusters as we published and validated in XP’21 [
The proposed model consists of five principles: (1) Embrace change to deliver customer value (2) Plan and deliver software frequently (3) Human centricity (4) Technical excellence and (5) Customer collaboration with no particular order of importance. Principles provide the frame of the assessment model, and are populated with clusters.
The proposed model consists of seventeen clusters. As the numbers suggests, clusters are not
enforced to be equally distributed across the principles. The clusters came into life as a result of
semantic grouping the identified agile software development practices in our literature study [
The questionnaire is the backbone of the model, as it defines the core of the evaluation. For each cluster there exist multiple questions, and sum of all clusters’ questions form the entire questionnaire. In order to systematically develop questions and answer option statements for each capability level-question pair, we setup a meta-structure as a guideline. Questions are therefore either related to specific aspects of artifacts. We also defined, how to bring in indicators, metrics or KPIs at which capability level of a question. We additionally decided to diferentiate from SPICE [ 8] and CMMI [9] to work with specific practices on all capability levels, and therefore to avoid generic practices at higher capability levels. The developed questionnaire contains the questions, capability levels along with their underlying definitions as well as the answer option statements dedicated to each capability level-question pair. The overall capability level definitions enables systematically defining the descriptions for each capability level per question. This additional layer of granularity is an important diferentiation point of our design from the existing models. This is because while other models are evaluating whether certain agile practices are performed in the organization, we are going beyond and evaluating the degree of performing those practices, and quality of the produced artifacts. As our model consists a total of seventeen clusters, any evaluation may easily expand to an impractical amount of questions for each cluster, making this a minimum spanning problem. Therefore, in order to increase the practical applicability of the proposed assessment model our aim is to reach the maximum coverage of agility by means of the minimum set of questions that allow us to maintain practical applicability without sacrificing quality. Due to space limitations, the questions of both the Technical Excellence and Human Centricity Principles are provided at https://bit.ly/3NG8glr.
Figure 1 visualizes a question from a cluster under Technical Excellence. In total, the validation covers eight diferent clusters consisting total of thirty two questions. For each question, the provided answer option statements per capability level brings in another level of complexity to the overall design with total of hundred sixty statements.
Remaining elements of the overall assessment method (e.g., assessment guidelines, aggregation and evaluation mechanisms) are mentioned under Section 8.2.
Our previous study [
Artifact orientation is one of the core guiding aspects of our model development approach. It is important because constructing an assessment model on artifacts contributes to the objectivity of the assessment results. In our model, the artifact orientation is reflected under the questionnaire.
Some artifact examples of the Technical Excellence principle’s Testing cluster are “Test Strategy”, “Test Plan” and “Unit Test”. If we take test strategy, while mere existence of a test strategy itself may hold value to an extent, the provided leveling mechanism allows identifying a range of states: whether the importance of having a test strategy is acknowledged, whether a test strategy is in-use up to the point where an existing strategy is being reduced down to its essentials and new, contextually supportive strategies are introduced. Evaluating the state of an artifacts not only supports the endeavour of conducting objective evaluations, but also provides a concrete path of improvement to the assessed entity.
Enabling self-assessments is another guiding aspect of our model development endeavor. This aspect is important mainly for two reasons. The first one is about the practical applicability of the assessment model. The questions are designed and validated during the design iterations, to ensure that they are understandable. In industry, assessments hold a great potential in providing units with a realistic image of their current state and actionable insights for improvement opportunities. With our model, we encourage units to conduct their own assessments and identify improvement areas with respect to their own targets. To realize that, we designed the questions of the questionnaire without any framework-specific terminology and provided guiding examples for each question. Second, the leveling mechanism has its own underlying descriptions to ease the assessment process during self-assessments. The aforementioned artifact orientation for objectivity is also a contributing factor to this aspect.
The modularity corresponds to the aspect of the assessment model that enables contextually appropriate assessments. To achieve modularity, we separate the concerns of diferent context defining elements in our design. For example, questions under each cluster are categorically labeled as being relevant only for large scale organizations. This gives our model the flexibility to discard contextually inappropriate questions, and to adapt the questionnaire size. Similar mechanism applies to other context defining elements such as having hardware development concerns, or having geographically distributed units. Having such modular structure enables the questionnaire to be tailored specifically to the changing organizational contexts.
In our journey towards establishing a practically applicable agile assessment model that is both academically and practically validated, we designed our research method as a design science research methodology as proposed by Hevner et al. [10]. In order to ensure the soundness of the model and questionnaire development activities, we structured our own research in the form of an agile process, where we conduct interview studies with the experts in our organization and integrate the feedback into the method development process. While the design science research methodology provides the overall frame of research, in order to integrate the learning outcomes of the academic and industrial validation steps, we iteratively realize design science research in alignment with action design research defined by Sein et al. [ 11]. By doing so, feedback is systematically collected from the potential consumers of the design artifact.
Sidky et al. [7] propose one of the early models for guiding organizations in their agile adoption journey. Authors define an agile adoption framework with two core elements: agile measurement index and four stage process. These two elements support organizations in adopting agile practices. The agile measurement index consists of agile levels, principles, practices and indicators. Five agile principles of the measurement index are derived from the Agile Manifesto and the practices populate the evaluation matrix. The four stage process utilizes the described agile measurement index. The model is explicitly highlighting the importance of tailorability of the ifve levels, by describing the challenges behind reaching a consensus on the assignment of practices to the levels.
Qumer and Henderson-Sellers [12] provide an agile solution framework for software. The model is built upon a conceptual agile model that is accompanied by an analytical tool consisting of four dimensions and an agile toolkit. The core of the method comprises the following five aspects: Agility, People, Process, Product, Tools, and an additional Abstraction aspect to reflect the agile software development methodology. While the agile toolkit consists of seven main components, the provided analytical tool focuses on the following four dimensions: Method scope, Agility characterization, Agile value characterization and Software process characterization . In order to support these two framework components, the authors propose the Agile Adoption and Improvement Model (AAIM) that is built on three agile blocks: Prompt, Crux and Apex and six agile levels. The study highlights the importance of supporting situation specific scenarios in software engineering.
Fontana et al. [13] suggest an agile maturity framework based on the complex adaptive systems theory. Authors emphasize ambidexterity as a fundamental attribute towards maturity and the described framework focuses on outcomes rather than agile practices. The fundamental positioning of people in the software development processes is made explicit, while the trade-of between exploitation and exploration is mapped to the trade-of between the specific outcomes and adopting new practices. As a result of their cross-case analysis, the following six outcomes were provided: Practices, Team, Deliveries, Requirements, Product and Customer. The study concludes by underlining the importance of allowing context-specific practices in the maturing process while considering the values of the Agile Manifesto.
This section discusses the results of the expert interview study conducted with 7 experts. The demographics of the experts can be found in Figure 2.
Each interview was designed as 45-60 minutes, per principle. When the area of expertise fell
more towards ethics, communication, collaboration and not necessarily to software engineering
on the implementation level, only the human centricity principle was evaluated. The data
collection was both in the form of likert scale questions, as well as open text for further
descriptions. Likert scale questions along with their codings as described below, represent the
quantitative results. These are provided in the form of heatmaps in the Figures [
In order to deliver the results in a coherent way, the following subsections will share the results regarding the questionnaire, human centricity principle and technical excellence principle under Section 5.1, Section 5.2 and Section 5.3 respectively.
The coding mechanism of results in the following figures will be as follows: Strongly Agree: 5, Agree: 4, Neither/Nor: 3, Disagree: 2, Strongly Disagree: 1, while E1 to E7 refer to the interviewed experts. For the cluster mappings of the Human Centricity Principle, we use Psychological Safety (HC1), Efective Communication (HC2), Unit Empowerment and Autonomy (HC3), Unit Collaboration (HC4) and Personal Growth (HC5). Same mechanism applies for the Technical Excellence Principle as, Design and Coding Practices (TE1), Testing (TE2) and Data Driven DevOps (TE3).
The overall results show that the relevance of the artifacts used in the questionnaire, and the coverage of the questions with respect to the clusters they are asked under, do difer. As it can be observed in Figure 3, all but one of the experts that are involved in software development on the level of implementation agree, if not strongly, that the provided artifacts are relevant for representing their associated clusters. On the other hand, as it can be seen in Figure 4, the coverage of the questions with respect to the cluster they are associated shows that the questions of Psychological Safety (HC1) cluster lack the expected coverage.
Promising results with respect to coverage show that the rules we used for developing the questions as described under Section 2.1.3 were reliable reference points, as judging the coverage of a cluster requires full understanding of the questions and their formulations.
Another encouraging result is regarding the evolutionary leveling mechanism, where all 7 experts agreed that the leveling mechanism is suitable for diferentiating between the capabilities of organizations with respect to agile software development methodology.
For the interpretation of Figures [
Results of our interviews show that not only the structural components, but also the provided questions for both principles have been separately evaluated and agreed to be providing suficient coverage for a practical evaluation as well. It is important to mention again that the evaluation is not on the questions only, but also on the statements we provide for each capability level(L1-L5) as depicted in Figure 1. Overall, only one expert(E4) disagreed to use Human Centricity Questionnaire. 5 out of 7 experts agreed that they would be willing to use one of the questionnaires in real life while 4 out of 7 stated the same for both of the questionnaires.
As it can be observed from Figure 5, questionnaire belonging to this principle, excluding one expert, received particularly positive feedback with respect to its industrial applicability potential. In addition to that, Figure 5 highlights the coverage of the Human Centricity Principle by the proposed five clusters was very well received by the experts. It can also be observed in the same ifgure, on the level of a principle, the mechanism for addressing the large scale development aspects with respect to human centricity found to be suitable by most of the experts. On the level of clusters, addressing the scaled aspects were evaluated as suficient by three experts for Psychological Safety and Efective Communication. For Unit Autonomy and Empowerment, and for Personal Growth, the decision of not including an additional question for addressing large scale was also found appropriate. Further, combining autonomy and empowerment concepts under one cluster instead of separately addressing them received positive feedback. As a side note, although keywords such as purpose, meaning, engagement or valuing someones work, were articulated during the discussions on the Unit Autonomy and Empowerment, the current placement of these keywords under the Psychological Safety was found appropriate.
Regarding Human Centricity Principle, “failure” was highlighted as an important missing aspect of Psychological Safety by one of the experts. Moreover, it was suggested for large scale product development organizations that if there exists a misalignment between teams with respect to the artifacts and methods associated with psychological safety, the questionnaire should be able to address those and make them visible to the members of the organization. Another remark with respect to Psychological Safety was the importance of capturing abstract notions such as trust, integrity and transparency. In terms of Efective Communication, the domain of communication engineering along with its features (e.g., structure, construct, time, type) was commented as a useful component that can be additionally considered. Importance of in person gatherings was also discussed in relation to COVID-19 related developments and their consequences. It was also reported that while tool support is very essential, the utiliziation of tools may difer among diferent scales of an organization. A critical view on Unit Autonomy and Empowerment was highlighting that the existence of organizational structures supporting autonomy do not necessarily mean that the organization can operate autonomously, as it relies on people and their intrinsic motivation to own certain tasks and work in an autonomous way. Additionally, even the individuals are intrinsically motivated towards self-organization and working autonomously, one of the experts commented that the resource limitations may hinder the process, even if the will is there. For Unit Collaboration, one of the expert’s recommendation was framing the collaboration aspect as an activity of mutual artifact creation and therefore suggested us to inspect joint eforts going into the creation of any software artifact such as revision repositories. When it comes to the Personal Growth, following two aspects were commented as important: first, how growth is perceived in the organization and second, the relevance of decoupling the views of the provider and receiver of any learning component.
As it can be observed from Figure 6, questionnaire belonging to this principle received particularly positive feedback with respect to its industrial applicability potential.
As a positive remark, except one expert, questions associated with each cluster of Technical Excellence Principle are found to be easy to understand. Moreover, regarding addressing the large scale product development on a principle level, Q2 shows that the suitability of the proposed mechanism is agreed by all experts. On the cluster level, the questions tailored for the scaled aspects of Design and Coding Practices was also found to be suficient in addressing the large scale development by all but one expert. The comment from that one expert was the necessity of including “interface and cross-component connectivity” under this cluster. When it comes to Testing, the scaled aspects received positive response from three out of five experts. The remaining two highlighted the importance of an “overall test architecture” and suggested “test strategy” to be explicitly considered also at large scale. Data Driven DevOps as well received positive response from three of the experts with respect to completeness, while the other two highlighted the importance of “purposeful measuring” and handling metrics and dashboards separately.
Clusters of the Technical Excellence Principle received additional comments regarding their coherence and completeness. Design and Coding Practices was primarily evaluated with high relevance and suficiently good coverage with its underlying artifacts and methods. Consequently, it did not received any negative feedback, however, one of the experts shared its own preference towards considering software architecture related aspects under this cluster more explicitly. Following comment was emphasizing the importance of reflecting the architectural decisions both on team and large scale product development level. Testing received additional input regarding the level of granularity of its questions. Two experts stated that the joint representation of two questions under this cluster may reduce the assessment time without compromising the quality of results. In terms of Data Driven DevOps, an improvement suggestion was towards the clarification of the “incident documentation” artifact. It was denoted by one of the experts that incident documentation may not be a very informative artifact, if the area of incident is not specified within the questions.
In order to avoid the potential risk of leading to human fatigue, the scope of this research focuses on the questionnaires of two principles. In order to deliver the results in a coherent way, the subsections will address the following: our findings that are not specific to either of the principles, findings that are specific to the human centricity principle and findings that are specific to the technical excellence principle, under Section 6.1, Section 6.2 and Section 6.3 respectively.
As the main finding, it is important to highlight that both questionnaires are found to be applicable and showed potential for being utilized in the industry. As mentioned in Section 5, questions Q3, Q4 and Q5 refer to experts’ opinion regarding their usage of this model for their own purposes and the responses to these questions are very promising and encouraging. Artifact orientation in the questionnaire was very well received, which is an important reference point and learning outcome for future researchers to consider.
On the other hand, understandability of the questionnaires were very high. Surely, there are suggestions and minor improvement possibilities for each principle. The results of the expert interviews show that the questions of the Technical Excellence Principle have a better coverage than the questions of the Human Centricity Principle. Here, the Psychological Safety stands out as a cluster with a relatively lower coverage. Our interpretation is that, as a very intangible and subjective concept, psychological safety becomes hard to evaluate on an artifactual basis.
Another important remark was on setting targets appropriately. The ability of establishing targets is very fundamental to any improvement activity, however, setting targets carelessly brings the risk of jeopardizing the optimization and continuous improvement activities. This is usually perceived as a hidden risk, as the individuals may believe that the organization is on its way to substantial optimization, without noticing that the optimization is directed towards a wrong, or sub-optimal target.
Besides the aforementioned generic findings, below we also address our research questions based on our findings.
RQ1: Do the questions belonging to Human Centricity and Technical Excellence principles cover the concepts related to the respective principles they represent? It is clearly indicated that the existing clusters suficiently cover the principles they are positioned under. This is a very important finding to ensure the reliable foundation of the structural components of the overall assessment system, as the model components provide the foundation for it.
RQ2: Is the proposed mechanism for addressing the large scale product development applicable from a questionnaire perspective? As agile at large scale is one of the contemporary areas of research, and as organizational scale is one of the context defining components of our modular approach, we provided a mechanism for addressing the aspects of clusters that are only relevant for large scale organizations. Our mechanism for addressing the large scale aspects of agility has received very positive feedback and was found useful by the experts.
Regarding human centricity, one of the essential comments was that the leadership, whether low, medium or high level, plays an important role in the organizations’ ability to act autonomously. This was suggested as an aspect that would, once made explicit, increase the coverage of the Unit Autonomy and Empowerment. In the model structure, notion of leadership is implicitly considered under Unit Autonomy and Empowerment, yet, is expected by the interviewees to be more explicitly represented. Another aspect that was frequently discussed during the interviews was the contrast between enabling the members of the organization, and actively supporting them.
Under Efective Communication and Unit Collaboration, it was also discussed that the technology or best practice sharing as well as the community of practice like initiatives increase the cohesion among the organizational structures, and contributes to the organizational synergy. Regarding the unit collaboration, a systematic enabler for an organization was mentioned as establishing rewarding mechanisms and objectives that are supportive of the collaboration across units, and not inhibiting their information experience sharing. To us, it is also clear that, once organizations establish the rewarding mechanisms and objectives in a way that they do not conflict with other units, there exist higher chances for collaboration to foster organically.
It is evident that the new ways of work, along with the transformations COVID-19 brought into our professional lives, shifted the medium of work to online tools and platforms. This came up particularly under Personal Growth in relation to the training and learning topics. It was commented that, be it technical or personal, the importance and efectiveness of on-site trainings should not be overlooked. It was discussed from the trainers point of view that tracking the learning progress of an individual becomes very challenging and observation based evaluation opportunities are lost in an online setting.
One particular aspect related to the personal growth was the consideration of time and motivation as fundamental resources for individuals’ growth journey. It was commented for many individuals, that it is not necessarily about whether there are enough training oferings or external resources for a learning task, rather it is about having the dedicated hours for a particular training, or about knowing the purpose and meaning behind the learning activities.
Additional discussion regarding human centricity was also in the direction of taking action, instead of just communicating the importance of a concept. With one of the experts, the necessity of making a distinction between communicating the importance of something and actively doing something was underlined, as this was perceived as an important aspect for capturing the actual state of agility being practiced and not staying only theoretical. This distinction can be made explicit by employing verbs such as enabling, realizing or instantiating.
The overall findings regarding the technical excellence principle were very positive and there were no major change requests.
From one of the senior experts from the testing domain, the assessment questionnaire was described as having a good structure for self assessments and positively commented on its ability to guide the individuals. Further, clusters were perceived as very valuable for representing the technical excellence, particularly in a software development organization.
In parallel, there was the discussion about the testing cluster’s level of granularity and coherence in terms of representing testing related activities such as unit testing, integration testing, system testing, their automation and systematic advanced applications. The conclusion was distributing automated testing and advanced testing methods as an element of higher level capabilities, rather than having a standalone question for them.
For technical excellence, another main discussion line was on the infrastructure and environment orchestration. When it comes to technical excellence and the quality of the software products, it was highlighted that along with the dominance of new architectures and technologies such as microservices, containerized applications and automated pipelines, the technical excellence requires organizations to have a new perspective on their delivery pipelines. Particularly at large scale product development, delivery pipelines and their automation configurations and testing activities have become a critical player in the value streams. As a result of that, the importance of paying as much attention to environments and delivery pipelines as the source code itself was stressed. Representative example could be having a containerized, shippable product, yet having failures in the container application itself.
An additional highlight under technical excellence is that, although monitoring and data collection for systems have a great potential in providing valuable insights, organizations need to identify what and why they are willing to measure certain indicators. If not, may result in overwhelming amount of data that is of little value and do not provide any actionable feedback onto system.
Another discussion point with respect to addressing the large scale product development concerns was the following: while the alignment of terminology, practices and methods across multiple teams is beneficial, it should be kept in mind that enforcing certain best practices onto other teams may result in reduced productivity. One should pay utmost attention not to fall into the alignment trap where the efort towards ensuring alignment starts negatively afecting autonomy.
Additionally, when it comes to addressing the aspects regarding the large scale, it is important to remember that while large scale product development may require handling totally new artifacts and methods, it may as well be the case that certain team level artifacts and methods may remain relevant in a slightly diferent form. An example to this could be, while data collection and monitoring are relevant for both on a team as well as for product level, the data to be monitored and questions that need to be answered are diferent for one team and for a large scale product development process.
In this section, we discuss the validity threats and our attempts to ensure a high quality of research by keeping these threats minimal. The initial consideration of the threats to validity have been the four well known threats as described by Yin [15]: Construct Validity, External Validity, Internal Validity and Reliability. These threats to validity have been adapted to the domain of software engineering by Runeson et al. [ 16]. Within the scope of this paper, as our methodological focus is not the case study research, we address three most relevant threats for our methodology.
Construct validity refers to the mechanism which entitles researchers to address the mutual understanding of the concepts being discussed and vocabulary being used. To address threats on construct validity, we have provided clear examples in our interview guidelines to establish the boundaries for discussion. Further, exactly the same examples and artifacts were used across each of the interviews. We also asked interviewees if the provided examples were helpful for them to have clarity of discussion. All experts validated that providing concrete examples within the questionnaire was helpful for them.
External validity refers to the generalizability of the generated results by the research activities. To address external validity threats, we consulted experts from diferent divisions, and put efort in gaining viewpoints based on geographical diversity. By doing so, our aim was to have a sample that is representative of diferent business strategies, work cultures and disciplines, yet, having seven experts partnering in our interviews remains as a threat to external validity. External validity is also afected by the fact that, although it is a very diverse conglomerate, all experts are within the Siemens organization, and may be expected to share common values and thinking patterns of the organization to a certain degree.
Reliability refers to the reproducibility of the research results. If the research have been conducted by diferent researchers, the same results should be derived. To ensure the reliability of our research, we included additional questions and objective interview guidelines for receiving feedback on the complexity of the interview. The interview results have been provided to the experts for an additional approval step.
This study follows on the previous two steps of our research [
Our aim in questionnaire development is to reach maximum coverage with minimal set of questions, by reducing complexity. The results of our interview study shows that not only the structural components falling under human centricity and technical excellence provide suficient coverage, the provided questionnaires both for human centricity and technical excellence have been evaluated as providing suficient coverage for a practical evaluation as well. Overall, only one expert disagreed to use one of the two questionnaires. Final and perhaps the most essential remark for us has been that 5 out of 7 interviewees agreed that they would be willing to use one of the questionnaires in real life while 4 out of 7 stated that they would be willing to use both of the questionnaires. As already mentioned in Section 2.1.3, the development of the questionnaire was carried out on the basis of guidelines we developed beforehand. The validation of the clusters of the two principles is a good indicator for us now, that the guidelines used for questionnaire development proved to be useful and will be used for the development of the questions for the remaining clusters and principles.
In the process of preparing this publication, we have completed the questions for the remaining clusters. First, we will be integrating the interview results in the questionnaire and implementing minor improvements as part of the intervention step of our action design research. Minor improvements regarding the example practices, and certain component name alterations will also be continuously considered. We will be finalizing our aggregation mechanism, as having an aggregation mechanism is important for being able to provide quantitative results out of the collected assessment data. Having this done we will apply the model in a pilot project in order to validate the overall suitability of the assessment model and method in an industrial setting by means of a case study. [6] T. Dingsøyr, T. E. Faegri, J. Itkonen, What is large in large-scale? a taxonomy of scale for agile software development, in: International Conference on Product-Focused Software Process Improvement, Springer, 2014, pp. 273–276. [7] A. Sidky, J. Arthur, S. Bohner, A disciplined approach to adopting agile practices: the agile adoption framework, Innovations in systems and software engineering 3 (2007) 203–216. [8] T. P. Rout, K. El Emam, M. Fusani, D. Goldenson, H.-W. Jung, Spice in retrospect: Developing a standard for process assessment, Journal of Systems and Software 80 (2007) 1483–1493. [9] C. Institute, Cmmi model v2.0 in retrospect: Developing a standard for process assessment (2018). [10] A. R. Hevner, S. T. March, J. Park, S. Ram, Design science in information systems research,
MIS quarterly (2004) 75–105. [11] M. K. Sein, O. Henfridsson, S. Purao, M. Rossi, R. Lindgren, Action design research, MIS quarterly (2011) 37–56. [12] A. Qumer, B. Henderson-Sellers, A framework to support the evaluation, adoption and improvement of agile methods in practice, Journal of systems and software 81 (2008) 1899–1919. [13] R. M. Fontana, V. Meyer Jr, S. Reinehr, A. Malucelli, Progressive outcomes: A framework for maturing in agile software development, Journal of Systems and Software 102 (2015) 88–108. [14] P. Mayring, Qualitative content analysis: a step-by-step guide, SAGE, 2021. [15] R. K. Yin, Case study research and applications, 1st Edition from 1984, Sage, 2018. [16] P. Runeson, M. Host, A. Rainer, B. Regnell, Case study research in software engineering: Guidelines and examples, John Wiley & Sons, 2012.