DevOps is a new tendency in business and information technology alignment. The purpose of DevOps is bridging the gap between the development and operations. Several sources claim that DevOps is a new style of work. Many successful DevOps introduction attempts and also many problems in adoption of this style of work have been discussed. This paper reports on research results in facilitating the adoption of DevOps in small enterprises. The DevOps adoption method and several related to it artefacts are proposed. The proposed method has been tested in a national branch of an international company with an internal IT development team.
Today, the business environment is very dynamic with rapid changes in different areas including IT. These rapid changes impact software development process, because customers have high expectations with respect to the content of applications and are demanding higher quality and shorter delivery times [1]. This is one of the reasons why agile practices are widely used and are recommended to be implemented by software development teams.
Still, usually, agile practices are used only on the development side; however, a software development process is not finished with the last acceptance-test and passing the whole deployment over to operations team [2]. Customers cannot use software which is not delivered to the production environment, ‒ and thus the business side cannot get value from the software even if it has been developed very fast. Based on Hüttermann [2], goals of the operations and development teams are different, part of them are even opposite to each other; for instance, the fast change delivery for the development team and the system stability for the operations team. This contradiction does not allow both teams working as one whole. According to reports [3, 4], the “deliver software faster” and the necessity for wider collaboration among IT teams are some of the drivers for adoption of DevOps. Such practices as useful and meaningful collaboration, shared ways of working, or shared goals and trust can lead to better collaboration and understanding between the teams and allow bringing agility into the maintenance part of software development process, too.
The other driving force for DevOps adoption, mentioned in reports [3, 4], is the use of less resources for software development and maintenance, which can be achieved through automation. It is the core activity in one of the DevOps practices - the continuous delivery [5]. DevOps adoption can help to solve the above mentioned and other problems; however, the adoption of DevOps is not trivial and can require complex changes in an enterprise process organization and workflows. To succeed in the DevOps adoption, the enterprises should understand different aspects related to the DevOps approach, have a clear strategy and a plan covering all relevant aspects. They should start the adoption process with the clear idea – what actions should be performed, how they should be prioritized, what tools could support these actions and how to measure the success of the adoption process. Therefore, in this paper, a dedicated DevOps adoption method is proposed for simplifying DevOps adoption in small enterprises. The method could help to save time of enterprises in DevOps adoption and be as a spring-board in the DevOps adoption process.
In order to conceptualize the DevOps adoption process and develop the DevOps adoption method the main research question – “What is the method for simplifying the adoption of DevOps?” was divided in several sub-questions including such questions as “What are the drivers of the DevOps adoption”, “What are challenges related to the DevOps adoption?”, and ‘What are the existing methods regarding to DevOps adoption?”. All sub-questions were answered with the help of systematic literature review thus providing the background for the DevOps adoption method’s development.
The paper is organized as follows: Section 2 introduces the challenges in DevOps adoption. In Section 3 the proposed method and some of its artifacts are presented. Section 4 reports on an experimental use of the method. Brief conclusions are provided in Section 5. 2
Knowledge of potential challenges, derived from available experiences of DevOps adoption, can allow avoiding problems during the adoption process. Even if it is not possible to avoid the problems, this knowledge can help to be more prepared for meeting the challenges. In some cases, the identification of potential challenges can even stop planned adoption processes, if the enterprise decides that it is not possible to overcome the challenges with a reasonable amount of resources.
According to Hamunen [6] DevOps adoption challenges can be grouped in four groups:
Lack of awareness Lack of support Problems linked to the DevOps technological implementation
Problems with adapting organizational processes to DevOps
In this section these four groups of challenges are further used for organized amalgamation of detected DevOps challenges.
Regarding the lack of awareness in DevOps the following challenges can be discovered:
Missing maturity of the concept – no clear definition of DevOps and its practices, no any clear goals available, and a lack of understanding about development workflow phases and responsibilities [3, 6, 7].
Buzzword perception/ allergy – many people perceive DevOps as a new buzzword, which is just the “last shriek of fashion” and will disappear as soon as has arisen, which does not allow perceiving this approach seriously [6, 7]. Lack of awareness – which includes missing communication, misinterpretation, insufficient knowledge and a lack of proper training [6, 7, 8].
The following three types of lack of support for DevOps adoption process can be identified:
Lack of management support – including difficulties to “buy-in” senior level management, which can lead to the lack of budget for DevOps adoption [3, 4, 6, 7].
Lack of team level support (or resistance) – can include either operations team resistance to changes or even development team resistance to changes because of the above described lack of awareness of DevOps [4, 6, 7, 9].
Lack of trust – including the lack of trust in DevOps idea and lack of trust in people, who promote and work on DevOps adoption process, because of the lack of understanding or missing/ insufficient communication. It can also be caused by just a fear of changes, a fear of potential failures, and a fear of measurements, which could indicate some unpleasant points [3, 6, 7].
Regarding challenges linked to DevOps technology implementation the following points can be explored:
Heterogeneous environment – including test environment limitations due to data management challenges, complexity due to multiple production environments and monolithic system architecture, which should be designed anew to support DevOps approach [4, 6, 7, 8].
Industry constraints and feasibility – because of legacy issues (not having access to production, etc.), compliance concerns, or security and sensitive data issues [3, 4, 7, 8, 9].
Application types – which is based on the situation that infrastructure does not support modern approaches and does not allow automatic deployments and integration of different tools [4, 6, 7].
Automated testing – based on worries about test script writing process integration in the code writing process and a support of it [6, 7].
Tool challenges – includes worries on how to choose the right tools and integrate them with existing tools and practices [6, 7].
Right scope for monitoring – based on worries about the amount of processes, data, applications, etc., which should be monitored to gather sufficient information for process support and potential improvements, but without putting big weight on the systems [6].
The following challenges with adapting organizational process to DevOps can be identified:
Geographical distribution of teams [6, 7].
Very complex organizational structure – with too many employees involved in software development process and too many interdependencies. If this is combined with deep-seated company culture with high resistance to changes, it can be a deal-breaker against DevOps adoption process [3, 6, 8].
Different team capacities – for instance, quality assurance and operations team capacity does not allow to manage development team capacity and deliveries; or business side stakeholders, who cannot support such frequent delivery cycles [4, 8].
Change management process integration – including existing software development method adaptation or introduction of new software development methods, new metrics adoption, and existing business process integration with the DevOps approach [6].
Right scope choosing – decisions of the scope can lead to either very good success of DevOps adoption process or vice versa; which puts these decisions to the challenges list [6].
Unlike other sources, Menzel [10] has looked on DevOps challenges from another viewpoint and has identified so called dual-challenges related to DevOps. They can be grouped as in-side and out-side challenges.
Out-side challenges are related to environment, which is around DevOps adoption process, and include the following challenges:
Wall of confusion based on working in silos, where for each silo (department) there are different goals and minimal information flows between them. Speed of innovations based on changes, necessary to be implemented in the systems.
Complexity of existing environments, where non-production environments do not reflect into production environments, which makes difficult to perform fast rootcause analysis.
Difficult error prevention and diagnosis process.
In-side challenges include:
Misunderstanding of DevOps approach and benefits that can be gained from it. Perceiving DevOps just as tool implementation, without taking into consideration collaboration, communication and other DevOps culture aspects.
Difficulties to manage current changes and DevOps adoption, because, usually, there is no option to stop business requests for system development and changes. Difficulties to choose the right adoption strategy – whether to choose the Big bang or Step-by-Step based (Phased) approach, as each approach has its pluses and minuses.
In the proposed DevOps adoption method, the existing DevOps adoption approaches, original DevOps maturity model and Nine impediment categories framework [11] are integrated with an aim to get a prioritized list of DevOps practices and tools to be applied in small enterprises. The constituents of the method will be discussed in more detail hereafter in this section (see Fig. 1).
The method consists of the following steps, included in the high-level DevOps adoption method’s model presented in Fig. 1: 1. Detect impediments to software development flow, based on framework of Nine impediment categories, prioritize them with the help of the questionnaire and get the list of relevant prioritized DevOps practices. 2. Establish the existing DevOps maturity level, based on DevOps Maturity model and select the desired DevOps Maturity level. 3. Get the list of prioritized DevOps practices, by summarizing priorities from the step of impediment identification and relevant DevOps practices, derived from goals, identified in DevOps Maturity model by the help of the identified gap between the existing and desired DevOps Maturity levels. 4. Identify tools existing in organization, related to DevOps approach, and get the list of related DevOps tools, supporting listed DevOps practices. 5. Choose DevOps adoption object for the first phase like a small start-up project, a small self-sufficient system or a tool, which will be a pioneer for DevOps adoption and will bring the most benefit for the team. 6. Identify related metrics for the DevOps adoption object, which will allow measuring the success of adoption process. 7. Process the first phases of DevOps adoption, collect and share results. 8. Start the next DevOps adoption phase by choosing the next DevOps adoption object (step 5) or by starting with step 1 and evaluating again impediments, existing maturity level based on changes that have happened; and set new priorities and related DevOps practices and tools.
Further in this section we will provide information about some of the artefacts mentioned in the description of the steps of the method. Information about all artifacts and guidelines for use of the method are available at the dedicated website at https://devopsadoptmeth.wordpress.com/.
The first artifact is the list of impediment categories that, together with the related questionnaire, is used to prioritize the areas where the enterprise wants to improve. “Nine impediment categories” framework includes the following categories [8]: 1. Work in progress – it is work, which is not completed and does not provide any value to the customer, who has ordered it. Usually such work is stuck in some of the phases of software development life cycle and is not delivered to production.
Similar to “Partially done work” type of waste. 2. Extra processes – these are activities that generate extra work, which consumes time and effort without adding value, like additional steps, barriers, documentation or reviews. 3. Extra features – development parts, which are added without proven need or validity. They take resources, which could be used for valuable features instead, or create a delay for some value-added features. 4. Handovers – activities related to incomplete work handing between persons or groups. Similar to “Task switching” type of waste.
Delays – related to situations, when one activity should be hold back because of waiting for other activity completion and delivery. Similar to “Waiting” type of waste.
Unnecessary motion – associated with avoidable movements of people, work or knowledge that create additional inefficiencies or disturb the smooth flow of work.
Failure demand – refers to the demand on system, team or organization, which is caused by own failure action, or action which is not done. Some similarities with “Defects” type of waste.
Context switching – related to situations, when people or team should divide their attention for more than one activity at the same time.
Unmet human potential – related to the waste of not using or fostering people skills and abilities to their full potential.
The next artifact represented here is the DevOps maturity model that was developed on the basis of analysis of related work and includes five levels of maturity with respect to the four enterprise areas, namely, technology, process, people, and culture. The maturity model is represented in Table 1. For each chunk in the maturity model, corresponding DevOps practices reported by DevOps practitioners were associated. Thus the enterprise can see which of these practices are or are not used in the as-is situation of the enterprise. Also the enterprise can see which practices shall be acquired to reach particular maturity levels, and decide on the desired level of maturity, which is realistic for the enterprise’s to-be situation. The table that illustrates the practices for the technology area of the maturity model is represented in Table 2. Similar tables are also developed for other maturity model areas.
ID
T1 Y G O L T2 O N H C E T
T3
Environments are provisioned manually Manual tests or minimal automation Data migration unversioned and performed manually
All environment configurations are externalized and versioned Functional test automation Changes to DB done with automated scripts versioned with application T4 T5 T6 T7 T8 T9 PR 1 PR 2 PR 4 PR 5 S S E PR CO 3 R P Manual deployment
Build automation Manual processes for building software/ No artifact versioning No collaboration tools No software configuration management (SCM) No or minimal monitoring No tools or minimal tool usage for issue tracking Inconsistent delivery process Ad-hoc development Ad-hoc testing Inconsistent project management Deployment and development documenta
Regular automated build and testing. Any builds can be recreated from source Project planning tool Standardized SCM Core monitoring All issue and bug reports are tracked Scheduled delivery process Scrum development Requirement based testing Project & requirement management Development documentation and relevant Nonproduction deployment automation Automated build and test cycle every time a change is committed Team/ toolset integration Configuration is delivered together with code Integrated monitoring Issue reporting automatization and monitoring Automated delivery process Agile development Integrated testing Integrated project management Regular validation of the documentation Production deployment automation Build metrics gathered, made visible and taken into account Knowledge management tool Self-healing tools Analytics/ Intelligence Activities based on received feedback and data Frequent delivery process Lean development Qualitative testing Quantitative proj. management Documentat ion process
and structure P1 P3
C1 E L P2 P O E P E C2 R U T L U C
C3 tion is not available or is out of date configuration files are upto-date Uncontrolled or reactive processes (not applied management) Teams organized around skillsets Ad-hoc learning Ad-hoc approach regarding competences development Restricted communicati on Uncommunic ated vision Lack of awareness of how culture is impacting day-to-day business
Processes are managed, but not standardized Team organized around deliveries Team learning Competences are developed with the help of training and development Rapid intrateam (inside) communicati on Clear delivery requirements Awareness of aspects in culture that may help or hinder dayto-day business and related configuration descriptions are provided Processes are standardized across organization Team organized around projects Value stream learning Analysis of exiting competences and future development Rapid communicatio n between teams (interteam) Clear project requirements Cultural traits that support business strategies have been identified update based on gathered experience and quality requirements Visibility & predictability of entire process & performanc
e Team organized around products/ business lines X-process learning Mentor usage Clear product/ business line requirements Culture viewed as an asset to be managed Frequent, collaborative Rapid communicati feedback on
Highly optimized & integrated processes Interdisciplin ary teams organized around KPIs External learning Continuous capability improvement Clear organization requirements C4 Poor, ad-hoc communicati on and coordination
Managed communicati on
Active collaboration Collaborati on based on process measureme nt, which allows to identify bottlenecks and inefficienci es – Subinnovating/ Innovations Innovation by Strategic C5 – no by necessity design innovation innovations Labels in Table 1: Empty model cells, in which relevant goals are not provided, are depicted using “–“, Opt.t – operations team, Dev.t. – development team.
For the purpose of better utilization of information represented in the tables, there are specific questionnaires made that help to identify the information according to the developed maturity model and the tables of related practices. They are available at https://devopsadoptmeth.wordpress.com. ID T1 T2 T3 T4 Infrastructure as
code Continuous monitoring Application monitoring Integrated change
management Constant, effortless communication and
collaboration Active Stakeholder participation T5 T6 T7 T8 T9 Collaborative development
Tools are an important part of the DevOps approach. They are tightly related to different DevOps practices, such as automation, monitoring or integrated configuration, and help to introduce these practices.
Based on seven different sources, namely, [12, 13, 14, 15, 16, 17, 18], eleven toolchain groups were identified. They are shown in the first column of Table 3. A specific table-like form of representation of toolchain groups and tools related to each group was established. This form of representation makes it more comfortable to identify necessary tools for DevOps practices chosen by enterprises. Having knowledge on toolchain groups and tools related to the chosen DevOps practices, the enterprises can see what investments will be needed to ensure the tool support for the desired levels of DevOps maturity. The correspondence between the toolchain groups and DevOps practices is not shown here due to space limits, – it is represented at https://devopsadoptmeth.wordpress.com.
The experimental application of the method was done in Company X that corresponds to small and medium sized enterprise definition as a medium-sized enterprise with ~220–250 employees. Its main business is not related to IT. There are more than 500 outside collaborators, too. If we look at the proportion of number of system users vs. number of IT department employees (19+2) it could be approximately 1:200, because multiple collaborating partners’ employees are also using organization’s systems. The IT department can be considered as a small enterprise. Company X runs the business line where one of the main drivers is the possibility to react fast on clients’ needs and environment’s changes. All core business processes are related to the organization’s back-end system, which is developed using in-house development resources. The main part of collaboration partners and agents are using organization’s front-end web based system, which is linked to the back-end system using middleware services. Company X has a web based self-services system for clients, too, which is linked to the back-end system similarly as collaboration partner systems. All web systems and middleware services are developed using outsourced developers. The similar situation is with organization’s web based CRM system.
At this moment, IT department is divided in four silos, where one of them is Maintenance (operations) team and the other three are development teams (shared service centers (SSC)), which are divided based on high level business lines. One of the SSCs is located abroad in a different country. There is a testing team with one quality assurance specialist per SSC and a Web system development process manager.
According to information, which was received during DevOps adoption method’s application process (the company representatives applied firs 6 steps of the method); Company X was already using some of DevOps practices, which had allowed reaching the second DevOps maturity level in Technology area. Still there was not enough progress in other three areas, where maturity level was only at the first level. As all four areas were similarly important for a successful change implementation, it was essential to work on other area’s maturity level rising.
Regarding the prioritized DevOps practices listed, one of the highest priorities for Company X was automated testing practice, which was important to fill the gap relevant to the chosen maturity level and was one of the practices relevant to highly rated software development flow impediments. Other highest level priority practices were collaboration practices (e.g. active stakeholder participation, shared responsibilities, knowledge sharing practices, etc.).
Some attention had to be turned to DevOps practices, which were with high priority level based on impediment identification, but regarding DevOps maturity level were already established in the company. For instance, build automation, which supports second Maturity level and was assigned as a practice, which was already used. Probably there was still high level of waste because the practice was neither fully established yet nor established in all development processes. Similarly it was with constant effortless communication and collaboration.
The application of the method shoved that Company X has already a good tool set, where some of them like Jenkins, Jira, and Confluence were usable for more than one DevOps practice support, which meant that there were space for growth without high additional investment. It was also identified that tools, relevant to DevOps practices, which could be useful for Company X during DevOps adoption process were Containerization tools like Docker, Mercurial/ Bitbucket, Rocker and Vagrant and DB handling tools like DBMaestro, LiquiBase, or RedGate.
During the experiment it was necessary to clarify some issues with the company; and some possible future elaborations helpful in method's application were detected. In overall the method was applied successfully and Company X was able to gain the needed information for decision making regarding DevOps adoption.
The application of the method showed that the proposed method can help to meet (at least partly) all four groups of DevOps adoption challenges discussed in Section 2. It can clarify the DevOps concept. With clear representation of different DevOps maturity levels and supplementary questionnaires, it can facilitate communication regarding management support for introducing new DevOps practices. Concerning technical challenges, it helps to identify new technologies that can be well integrated in enterprise infrastructures. The same applies to adapting organizational processes to DevOps as the method helps to choose the right scope of steps to be performed in DevOps introduction at chosen levels of maturity. While the method does not address every single challenge mentioned in Section 2, by addressing part of them, it can positively influence enterprise ability to meet other challenges, too. 5
The purpose of this research was to design and validate the DevOps adoption method to guide small enterprises in DevOps adoption process and also to simplify the adoption process.
According to the DevOps adoption concept and relevant information summarized during literature review, which allowed identifying main points leading to the beginning of the DevOps adoption process, the design of the DevOps adoption method was started. The necessity to determine the existing situation in an enterprise and define the desired level of maturity was identified. Therefore the DevOps Maturity model was designed. The model was related to amalgamated DevOps practices and tools. The “Nine impediment categories” framework was used as to identify necessary DevOps practices, which a particular company could use during the DevOps adoption process in order to achieve the desired level of DevOps Maturity. To provide access to the method's description and its supplementary artifacts the web page was created at https://devopsadoptmeth.wordpress.com/. This page was used during method validation process described in Section 4 to provide easy access to the developed questionnaires.
The DevOps adoption method validation, performed with the help of a particular company’s IT team showed that the method allows to determine the list of prioritized DevOps practices and allows to derive the list of tools supporting these practices. After choosing the DevOps adoption object and establishing relevant metrics, an enterprise can proceed with the DevOps adoption process with the help of the derived DevOps practices and the identified list of supporting tools. The use of the method can help to meet several challenges of DevOps adoption mentioned in Section 2.
The method provides an opportunity of simplification of DevOps adoption in small enterprises. It has not been analyzed yet whether it is also useful for large enterprises.
The further research concerns application of the method in more companies, the DevOps adoption management tool development, the method's tuning; and the development of the approach for systemic artefact maintenance to ensure that all constituents of the method are continuously up-to date and consistent.
Acknowledgment. This work is supported in part by the Latvian National research program SOPHIS under grant agreement Nr.10-4/VPP-4/11.