The constantly evolving technological and business requirements of the market pose a multitude of challenges for modern development companies, particularly when it comes to developing and maintaining software systems. DevOps was introduced as a solution to these challenges, ofering a set of tools, rules, and practices that facilitate more eficient software development and maintenance throughout the development and production stages. DevOps is a software development practice that helps IT organizations perform better while delivering software products. By breaking down barriers between development and operation units, DevOps accelerates the development process. While implementing and adopting DevOps in an organization, critical success factors play a role in understanding what issues are important for achieving success in projects. Assessing the success factors for improved DevOps practices is crucial. Therefore, this study has been developed using Grounded theory and Gioia methodology to create a theory and model that combines DevOps organizational practices and tools to enhance DevOps core concepts and values.
DevOps is a widely used software development strategy that is used for minimizing
development costs through implementation and adoption practices. DevOps makes the software
development process eficient by providing continuous delivery and continuous development
support for both teams [
In contrast, DevOps is the combined process of ‘development’ and ‘operations’, which is used for software development to speed up the delivery process with eficiency [ 9, 10].As a response to the changing business environment, the software development life cycles and the development processes have been evolving. For the software development life cycle,success factors play a huge role in DevOps performance in the organization [8]. According to Bullen (1981), Critical success factors are those factors where favorable outcomes are required from key areas of activities and that are necessary for reaching the management goals [11].
The study has three main objectives that motivated its research. The primary objective was to understand the critical success factors for DevOps reported in the existing research literature. This objective aimed to provide a comprehensive overview of the key factors that drive successful DevOps projects and identify any gaps in the current literature.
The secondary objective was to understand the perceptions of professionals regarding DevOps critical success factors. This objective aimed to gain insight into the perspectives of DevOps professionals and their experiences with implementing DevOps practices. It also aimed to identify any diferences between the critical success factors reported in the literature and those perceived by professionals.
The third objective of the study was to identify the challenges of DevOps practices and risk mitigation strategies proposed by professionals. This objective aimed to identify any obstacles or challenges that organizations face in implementing DevOps practices and to explore potential strategies for mitigating these risks. It also aimed to provide insights into the experiences of professionals in addressing these challenges and the efectiveness of diferent risk mitigation strategies.
The study aims to address four significant gaps in the existing literature related to DevOps projects. very few studies have proposed a synthesized and validated framework that provides a clear and comprehensive view of the critical success factors of DevOps projects. This gap impedes the ability of organizations to identify the key factors that drive successful DevOps projects and hinders their ability to implement DevOps practices efectively.
Most literature on DevOps has focused on technical factors, such as automation, continuous integration, and deployment. While these factors are essential, they do not provide a complete picture of DevOps success. This gap in the literature limits the understanding of the nontechnical factors that impact DevOps success, such as teamwork, communication, and leadership.
Very few studies have investigated organizational factors that impact DevOps success, such as managerial practices, organizational culture, and change management. This gap in the literature limits the understanding of how organizations can create a culture of continuous improvement and implement DevOps practices efectively.
Finally, very few studies have applied grounded theory approaches to develop a framework for critical success factors. Grounded theory approaches are valuable in identifying key themes and factors that drive success in complex systems such as DevOps. This gap in the literature limits the ability to develop a comprehensive framework for critical success factors that can be applied in various organizational contexts.
By addressing these four gaps, the study aims to provide a synthesized and validated framework that provides a clear and comprehensive view of the critical success factors of DevOps projects. This framework will incorporate technical and non-technical factors, as well as organizational and managerial factors, and will be developed using grounded theory and Gioia approaches. The study’s results can be used by organizations to identify the key factors that drive successful DevOps projects and implement DevOps practices more efectively.
In recent years, literature review articles were published that described DevOps concept, DevOps adoption, DevOps Implementation, DevOps specific use, DevOps implementation with agile, and DevOps challenges, benefits and success factors [ 12, 13, 14, 15, 16]. Before the research we could not find any article that proposed a synthesized framework which could provide a clear, comprehensive idea about the critical success factors of DevOps projects.
To address this we have one key research question(RQ). The key research question is: RQ What are the critical success factors of DevOps?
To understand the key research question, we need to understand three sub-research questions. The sub-research questions are written below.
RQ1 What are the critical success factors, as reported in the extant research literature of
DevOps? RQ2 What are the critical success factors of DevOps as reported by professionals? RQ3 What are the challenges professionals face in DevOps projects? How do they mitigate the challenges and risks?
At first, we conducted a Systematic literature review by selecting empirical papers on DevOps practices. We have also conducted a multivocal literature review to understand the diferences between white and grey literature. Our aim was to understand how a multi-vocal literature review can be used to determine Critical Success Factors (CSFs) impacts on DevOps performances and practices and does it aligns with Grey literature. Then we used a mixed-method approach for data collection.
We have developed a questionnaire based on specific categories of DevOps practices and conducted an open-ended online questionnaire survey with 92 DevOps professionals. After sorting, 72 responses were deemed usable for data analysis. The study’s results provide valuable insights into the perceptions of DevOps professionals regarding various categories of DevOps practices,which can be used to inform the development of efective DevOps strategies and practices.
To gain a clearer understanding of the topic, the study conducted semi-structured interviews with 40 DevOps professionals from around the globe. These interviews provided the researchers with an excellent opportunity to gain insights into how DevOps practices are operated within teams. Through the interviews, the study aimed to gain insights into the perspectives of DevOps professionals regarding practices, collaboration, and risk mitigation strategies implemented in IT organizations for accelerated software delivery to market. The results of these interviews provided valuable insights into the team’s DevOps capabilities, including their strengths and areas for improvement. 5.1.3. Survey In order to measure the constructs presented in the critical success factors model illustrated in Figure 1, we have developed a 5-point Likert scale questionnaire. To this end, we have conducted a structured survey to collect approximately 140 responses, which we have utilized for our research purposes to validate the Success factors model.
For the SLR data analysis, We have followed the methodology by Kitchenham and Charters [17] for searching the research articles. After the search, we got the initial list of articles of 38 and our ifndings suggested that several critical success factors impact on organizational performance.
For the open-ended questionnaire survey with 72 DevOps professionals, we have employed the Gioia method proposed by Gioia et al. [18]. Gioia method suggested that it is important to investigate diferent actors based on events that could lead to novel findings of the research process. Our analysis followed an iterative process that involved the repetition of steps for data analysis. We have used open coding suggested by Strauss and Cobin. ( 1998) [19]. We have made constant comparisons so that we can have better codes from the survey data.
For the data analysis part we have used the thematic mapping method. According to khan et al. [20] thematic analysis is a method that helps to identify, analyze, and report themes and patterns within the data. It helps to understand the dataset in an organized way and interprets many aspects of the research topic [21, 20].
After that, we conducted 40 interviews with DevOps professionals to understand the professional perspective of DevOps practices with critical success factors. For the data analysis, we have used the approach of grounded theory and followed the data analysis procedures [22]. We have used open coding and constant comparison for data synthesizing. From our interview dataset, we have identified the patterns. According to Glaser in Grounded theory, open coding refers to a situation where the researchers keep their minds open while doing the data analysis [23].
The research is going to conduct a thorough study based on Software companies’ and IT professionals’ practices on DevOps use. We are going to test the success factors model- and hypothesis with PLS regression. We aim to provide a better understanding of the critical success factors of DevOps projects by identifying how those success factors impact DevOps implementation and its success [24].
In the paper titled "DevOps critical success factors — A systematic literature review,"
the authors conducted searches in four major publication databases and utilized
snowballing techniques to select 38 primary studies for analysis. The analysis
identiifed nearly 100 diferent critical success factors (CSFs), which were then categorized
into three dimensions: Technical, Organizational, and Social and Cultural.
Based on the results of the literature analysis, the authors proposed a comprehensive
framework that depicts how these CSFs impact or drive DevOps success. This
framework provides a clear and comprehensive view of the key factors that drive
successful DevOps projects, incorporating technical, organizational, and social and
cultural dimensions [
Paper 2: Understanding DevOps critical success factors: Insights from professionals.
The present research paper delves into the factors that contribute to the success of
software development achieved through the implementation of DevOps practices.
DevOps is a collection of practices that aims to streamline software development
processes by removing barriers between operation and development teams. Despite
the widespread availability of information on DevOps practices, adoption, and
challenges, there has been little research focused specifically on the critical success
factors of DevOps. This paper aims to address this gap by analyzing and discussing
the key factors that are vital for success in software development with DevOps. To
achieve this, we conducted an open-ended survey among 72 DevOps professionals
and employed the Gioia method to elaborate on the opinions of the professionals
on the identified success factors and connect them to previous literature.
The success factors that emerged from the study include intra-organizational
collaboration, organizational hierarchy, strategic planning, team dynamics, cultural shift,
performance engineering, integration, build and test automation, infrastructure,
and DevOps as a service. Based on the findings, we propose five DevOps
implementation advice that could aid companies while implementing DevOps practices.
These include management support, investment in DevOps tools, sharing
knowledge within teams, sharing responsibility in teams, and being willing to explore
and experiment with the practices while being agile. These factors are crucial for
DevOps performance and organizational success [
Paper3: Understanding DevOps critical success factors: A grounded theory approach The DevOps culture and practices have become ubiquitous in the software development industry. The primary objective of DevOps is to bridge the gap between the operations and development teams. The use of DevOps tools, company culture, collaboration, and employee skill sets are critical considerations for any company. The implementation of DevOps can present various challenges and success factors that must be taken into account. The critical success factors are essential to the company’s performance. To better understand and identify the DevOps best practices and critical success factors, we have employed the Grounded Theory approach. Our research aims to present a theory, based on interviews from IT organizations and professionals, providing detailed scenarios of DevOps practices. Our findings indicate that a collaborative culture, DevOps automation, working ethics, remote teamwork issues, and DevOps security teams are critical for better DevOps practices in organizations. We believe that this study will make a valuable contribution to DevOps practices by providing real scenarios that explain diferent categories of DevOps critical success factors, implementation, adoption, and performance from a broader perspective.
Paper 4: DevOps Challenges and Risk Mitigation Strategies by DevOps professionals
teams
DevOps is considered a team culture and organizational practice that eliminates
ineficient automation and bottlenecks in DevOps infrastructure. Various
companies are adopting DevOps nowadays. However, the concept of adopting DevOps in
companies is still challenging and risky. There is a need to understand risk factors
and the way of mitigating risks with strategies for practicing DevOps in the
organization. We have conducted 26 interviews with DevOps professionals to understand
the diferent challenges and risks the company faces while adopting DevOps in
organizations. Our findings suggest that there are several risks for adopting DevOps
practices. Those include a lack of ideas about the project, lack of tacit knowledge,
improper allocation of budget for the toolset, miscommunications between clients
and developers, improper code review by team members, reluctant to accept the
organizational culture, risky change and development, Security vulnerabilities in
DevOps pipelines, not using automated testing for quality assurance, human
errors in the system, a security breach in DevOps environment, mishandling ethical
issues for teams, DevOps practices misalignments with teams ethics and values,
not involving users and stakeholders in ethical decision making, Unethical data
collection from users, etc. As a risk mitigation strategy, diferent measures could be
taken by companies. According to the research findings some of the risk mitigation
strategies could be continuous testing, using infrastructure as a code (IAC),
Security audit and monitoring, disaster recovery planning, cross-functional training,
proper documentation, continuous learning, Continuous improvement, making
the process visible to the team members, prioritize personnel so they feel valued,
enforce security policy, introduce DecsecOps model, involvements of experts from
outside, improved management strategies. The list of risks and mitigation strategies
thus will help future researchers and practitioners to understand DevOp’s ongoing
practices in companies with a wider view and perspective [
This research timeline started in the year 2021, at LUT University, Finland. The author is planning to finish the study at the proposed time by 2025.
This thesis will contribute to several fruitful avenues for future work in the domain of DevOps. First, we observed that prior research lacks theoretical frameworks when investigating DevOps success. In this thesis, we will focus on the theory-guided approach to validate the proposed model (Figure 1) which is one of the novelties of this thesis.
Second, we will validate the success factors through semi-structured interviews with IT professionals. This will be an addition to the knowledge sector.
Third, we will develop and design scales for measuring the construct we have identified from our study and that will be an addition to the knowledge sector.
Fourth, we will also observe the viewpoints of Dev and Ops and do they vary for critical success factors, this findings will be an addition to knowledge work.
Fifth, many of the factors identified in our study will be an addition to the knowledge sector.
Sixth, our identified success factors can be used as a body of knowledge for the practitioners for better implementation of DevOps in the organization.
Finally, these key categories can help practitioners to implement successful DevOps implementation in projects.
The study primarily focused on the development and operations teams of organizations and utilized a mixed-method approach, encompassing both qualitative and quantitative research methods, such as semi-structured surveys. However, due to time constraints, the researchers could not conduct interviews with the whole DevOps team from each company. Though 40 company professionals participated in the interview sessions as their company representatives.
The study highlighted the importance of sharing common psychological aspects of operating in a team as a key aspect of DevOps culture. However, there is a need for further research to investigate the psychological factors that impact teamwork and organizational performance.
Moreover, the study also revealed that diferent organizational and managerial approaches could afect the handling of DevOps operations. As such, future research could explore these aspects to provide insights into the best practices for managing DevOps in organizations. 10. Future studies There could be more research scopes that are unanswered for this study. We believe that there could be more areas to focus on, and the scope of the study is thus limited. 10.1. Focus group Interviews However, to carry out additional research the author considers it essential to conduct case studies and focus group interviews for future studies. These two data collection strategies could be used to get more perspectives from the professionals. By taking a case organization it is possible to have a more depth understanding of DevOps practices and performances. 10.2. Measuring Psychological factors of team members with scales We can develop new scales and develop a questionnaire to measure team performance and the psychological factors of team members. At the same time, we try to understand more ethical ways of working in teams and the customer usage behavior of the systems in software production. 10.3. AI and DevOps Integration Unlock the potential of automated decision-making, predictive analytics, and anomaly detection in DevOps processes with the integration of artificial intelligence and machine learning. For future research Discovering the transformative power of this cutting-edge technology today is required. 10.4. DevOps in Regulated Industries In today’s fast-paced and competitive world, the implementation of DevOps is a crucial factor in ensuring the success of any organization. However, in highly regulated industries, security and compliance concerns can often be a major roadblock. As a future research direction investigate and adopt the best practices for implementing DevOps in such industries, ensuring not only the smooth functioning of the organization but also adherence to regulatory requirements. By adopting these best practices, the organization can achieve higher levels of eficiency, agility, and innovation, while also maintaining the highest levels of security and compliance. 10.5. DevOps cultural Matrics for collaboration in teams DevOps teams can revolutionize their collaboration and innovation capabilities by efectively measuring and assessing cultural aspects. Discover new and advanced ways to evaluate the cultural elements within teams to drive enhanced collaboration and innovation will be a good research topic for the future. Investigating ethical implications related to automation, AI, and decision-making within DevOps teams is crucial to understanding. 11. Conclusion The study utilized a mixed method approach, combining qualitative and quantitative research methods, including semi-structured interviews, open-ended questionnaire surveys and structured surveys to assess the impact of DevOps practices and critical success factors on overall performance in the software development process. The research examined various aspects of DevOps software development in IT organizations, gathering insights from 40 global DevOps professionals through interviews, 72 open-ended survey, and 130 structured surveys to understand DevOps professionals’ perspectives on practices, collaboration, and risk mitigation strategies implemented for faster software delivery to market.
However, the team encountered various challenges, risks, and ethical issues in gathering feedback from production and integrating practices. There were challenges to getting feedback from task management, stakeholders, and continuous improvement. Through the process, the researcher gained a deeper understanding of the topic and developed a theory of critical success factors. After CSF model validation that will contribute to the future identification of success factors in organizations and will help consultants to know the current DevOps trends and practices in organizations. 12. Acknowledgments The research work is supervised by Professor Sami Hyrynsalmi from LUT University. I am grateful to my supervisor and LUT University for their endless support. I would like to show my gratitude to all DevOps professionals around the world who have shared their valuable insights, and experiences and participated in the interviews and surveys. [8] N. Azad, S. Hyrynsalmi, What are critical success factors of devops projects? a systematic literature review, in: X. Wang, A. Martini, A. Nguyen-Duc, V. Stray (Eds.), Software Business. 12th International Conference, ICSOB 2021, Drammen, Norway, December 2–3, 2021, Proceedings, Springer, 2021, pp. 221–237. [9] J. K. Leidecker, A. V. Bruno, Identifying and using critical success factors, Long range planning 17 (1984) 23–32. [10] R. W. Macarthy, J. M. Bass, An empirical taxonomy of devops in practice, in: 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), IEEE, 2020, pp. 221–228. [11] C. V. Bullen, J. F. Rockart, A primer on critical success factors (1981). [12] W. Hussain, T. Clear, S. MacDonell, Emerging trends for global devops: a new zealand perspective, in: 2017 IEEE 12th International Conference on Global Software Engineering (ICGSE), IEEE, 2017, pp. 21–30. [13] R. Jabbari, N. bin Ali, K. Petersen, B. Tanveer, What is devops? a systematic mapping study on definitions and practices, in: Proceedings of the Scientific Workshop Proceedings of XP2016, 2016, pp. 1–11. [14] N. Kerzazi, B. Adams, Who needs release and devops engineers, and why?, in: Proceedings of the international workshop on continuous software evolution and delivery, 2016, pp. 77–83. [15] A. Kumar, M. Nadeem, M. Shameem, Prioritization of devops maturity models using fuzzy topsis, in: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering, 2023, pp. 438–443. [16] M. Muñoz, M. Negrete, J. Mejía, Proposal to avoid issues in the devops implementation: A systematic literature review, New Knowledge in Information Systems and Technologies: Volume 1 (2019) 666–677. [17] M. Turner, B. Kitchenham, P. Brereton, S. Charters, D. Budgen, Does the technology acceptance model predict actual use? a systematic literature review, Information and software technology 52 (2010) 463–479. [18] D. A. Gioia, S. D. Patvardhan, A. L. Hamilton, K. G. Corley, Organizational identity formation and change, Academy of Management Annals 7 (2013) 123–193. [19] A. Strauss, J. Corbin, Basics of qualitative research, Sage publications, 1990. [20] M. U. Khan, S. Sherin, M. Z. Iqbal, R. Zahid, Landscaping systematic mapping studies in software engineering: A tertiary study, Journal of Systems and Software 149 (2019) 396–436. [21] V. Clarke, V. Braun, N. Hayfield, Thematic analysis, Qualitative psychology: A practical guide to research methods 3 (2015) 222–248. [22] K. Smolander, M. Rossi, S. Pekkola, Heroes, contracts, cooperation, and processes: Changes in collaboration in a large enterprise systems project, Information & Management 58 (2021) 103407. [23] H. Heath, S. Cowley, Developing a grounded theory approach: a comparison of glaser and strauss, International journal of nursing studies 41 (2004) 141–150. [24] N. Azad, Developing a critical success factor model for devops, in: International Conference on Product-Focused Software Process Improvement, Springer, 2022, pp. 662–668.