Complex project management: using complexity and volatility to guide hybrid methodological practices

Abstract

Organisations adapt to their changing business environment by addressing problems of increasing complexity and volatility across both technical and social dimensions, with projects often the instruments used to do so. With widespread project failure, there is growing recognition that better Project Management (PM) methodologies are needed to deal with these problem characteristics, with practitioners resorting to ad-hoc hybrid PM practices to ll in the gap. Insights into such practices remain sparse. This context drives our research, whose overarching aim is to provide a theoretical and methodological basis for systematic PM hybridisation driven by organisational problem characteristics. In this paper, we focus on the relation between those characteristics and PM methodologies with insights gained through both secondary and primary research.

Keywords

• eol>hybrid project management
• socio-technical problem solving
• complexity
• volatility

-

1. Introduction The Project Management Institute (PMI), an international body aimed at bringing professionalism to the area of project management [1], sees projects as the instruments used by organisations to “help meet their strategic goals...[and] adapt to the changing business environment” [2]. In the context of heightened competition and ongoing technology, market and social disruption, the strategic value of projects is rising [ 3 ], together with a recognition of the relationship between business success and appropriate project implementation [ 4 ]. In particular, the correlation between project success and choice of Project Management (PM) methodology has been evidenced in the literature: for instance, based on a large-scale survey, [ 5 ] estimated that it accounts for 22.3% of the variation in project success, when measured against the overall project objectives.

In spite of the many PM methodologies, projects still fail with severe consequences for organisations. As recently reported by [ 6 ], 11.4% of investment is wasted due to poor project performance. The reality is that the problems faced are becoming more complex and volatile across social and technical dimensions, so that existing PM methodologies and practices are increasingly challenged. As a result, some authors have proposed that PM should be regarded as a form of organisational problem solving [ 7 ], while practitioners are looking creatively at combining predictive and adaptive features into hybrid PM approaches [ 8, 9 ].

Yet, an understanding of hybrid PM approaches remains limited, with few studies focused primarily on software product development (e.g., [ 10 ]), and any theoretical and methodological underpinning is lacking [ 11 ]. Addressing this gap is the overarching aim of our research.

There are two related facets. Firstly, the interpretation of PM as problem solving allows us to explore how to leverage an existing problem solving framework, called Problem Oriented Engineering (POE) [ 12 ], to provide the required theoretical basis for the work. With its roots in socio-technical design and engineering, POE has a track record of successful application to real-world socio-technical problems, including the engineering of information systems (e.g., [ 13, 14 ]). Its three-ellipse model for sociotechnical problem solving [ 15 ] and related POE problem solving pattern [ 12 ], balance the social and the technical in their wider real-world context, explicitly addressing the interaction between the two in the creative design process, in line with the socio-technical tradition of [ 16 ].

Secondly, the considerable body of knowledge on established predictive and adaptive PM methodologies provides a rich source for our secondary investigation into their strengths and weaknesses with respect to complex and volatile organisational problems, which we complement with primary research into current hybrid practices across industries.

After outlining our overarching research approach (Section 3), we focus on key ndings from our secondary research and practitioner survey (Section 4), then brie y discuss how they can inform a methodological approach for PM hybridisation (Section 5). 2. Background A project is “a temporary endeavour undertaken to create a unique product, service or result” with project management (PM) “the application of knowledge, skills, tools, and techniques to project activities to meet the project requirements” [2], usually combined into speci c PM methodologies.

PM methodologies have evolved since the 1950s to ensure robustness and applicability to a wide range of projects, to manage risk and maximise project success [ 17 ], from early predictive methodologies (e.g., the Waterfall), characterised by fully planned rational processes, to the latest adaptive methodologies, (initiated by the Agile movement) better equipped to deal with ongoing organisational and environmental change.

Budget, schedule and output quality have been the most commonly applied criteria to measure project success for decades [ 18 ], further quali ed into process performance (related to e ciency and adherence to schedule and budget [ 19, 20 ]) and product performance (related to the project outcome – scope – and realised stakeholders’ bene ts [ 21, 19, 22 ]).

Despite the many PM methodologies available, project success remains hard to achieve. For instance, a study [ 23 ] into 5,400 Information Systems projects, an important and pervasive class of socio-technical organisational projects, concluded that 50% of them either exceeded their planned budget before completion or addressed a reduced scope, while 20% even put the existence of the company at risk. In general, project performance across di erent organisations and industries remains patchy [ 22 ].

Project success is a ected by project risk, consisting of the likelihood and impact of negative events on a project [2], so that some form of risk management is a key element of all PM methodologies, aimed either at decreasing event likelihood through risk treatment, [ 24 ] or impact on the project through risk adaptation, [ 25 ]. The former uses controls – process, policy, devices or other practices or actions – to lower likelihood [ 26 ], the devices we rely on in this research.

Di erent methodologies deal di erently with contextual characteristics, due to their diverse ways of handling risk and uncertainty [ 27 ]. In predictive methodologies, a project is organised in fully planned stages, with gateways between them to maintain control and prevent ‘spillovers’ from one stage to the next, with the assumption that scope, schedule and budget are xed and entirely known from the start. Instead, in adaptive methodologies, a project is organised in iterative cycles in which retrospective reviews are used to learn lessons from one cycle to the next, so that scope, schedule and budget can be adjusted as the project unfolds. Predictive methods assume stability and predictability from the start, while the adaptive methods can cope with a dynamic environment where many factors may change. From a knowledge perspective, [ 28 ] describes predictive methodologies as “a picture of already knowing,” and adaptive ones as “a picture of learning,” with the assumption that the more novel the problem or its solution, the more the problem solving involves learning.

In a quest for a better t between PM methodologies and organisational problems, the last decade has seen an increase in hybridisation, with project teams attempting to reap the bene ts of combining the discipline of predictive methodologies with the exibility of the adaptive ones. However, little is known about hybridisation, with methodological support still lacking [ 29, 8 ].

In our work, we take a risk-driven approach to hybridisation by matching characteristics of organisational problems to speci c features of PM methodologies to increase the likelihood of project success, and to provide both theoretical and methodological support for PM practice. 3. Research methodology overview As the theoretical and methodological basis for hybrid PM, we adopt an existing problem solving framework, called Problem Oriented Engineering (POE) [ 12 ], with roots in socio-technical design and engineering. POE allows the expression of design and engineering problems and of step-wise processes for their solution. Its logical basis allows us to bridge between formally and non-formally described elements of a problem, an essential feature of socio-technical problem solving, in which both technical and human factors are intrinsic elements of the process [ 30 ]. Central to POE and its core (design) process pattern is the concept of stakeholder validation, as the ultimate arbitration of the tness for purpose of both the framing of a problem and the design and construction of its solution: validation both provides the means to ‘democratise’ the problem solving process [ 30 ] by involving end-users and other a ected stakeholders, as well as managing risk, a key component of PM.

Our overarching research methodology, which be ts our aim to de ne a novel methodological framework for hybrid PM, is that of ‘design and creation’ [ 31 ], via cycles of problem recognition (through secondary research and a practitioner survey), design and implementation of tentative solutions (from interview ndings, mapping between POE and PM, and exploratory case studies), and their evaluation (through case studies). In this paper we focus on problem recognition and survey/interview ndings informing an initial solution design.

As our research is concerned with phenomena in a social setting, i.e., the organisation, we follow the interpretive research paradigm [ 32 ] aimed at analysing social processes and how human agents make sense of their perceived worlds, with the rst author, an experienced project and programme manager of over 20 years, shaping the research process through re ection on hers and their colleagues’ knowledge and experience, in line with the tradition of action research in socio-technical design [ 30 ].

In compliance with the validity criteria of the interpretative paradigm [ 32 ], objectivity and reliability is ensured by appropriate triangulation between secondary and primary research, the use of several data generation methods and by documenting all research procedures so that an audit trail can be carried out. A degree of external validity is achieved by means of transferring the ndings between di erent organisational settings. Social complexity social [ 28 ]; socio-political (conflicting stakeholders’ interest and di icult personalities); uncertainty (lack of agreement) [ 11 ]; environmental (political influences) [ 34 ]; organisational [ 35, 36 ] Social volatility volatility (target, governance) [ 37 ]; volatility (requirements) [ 38, 39, 36 ]; dynamics (change in management team or environmental context) [ 11 ] 4. Key findings from secondary research and practitioner survey Our secondary research had the broad aim to investigate characteristics of organisational problems as well as how di erent PM methodologies deal with such characteristics and which speci c risk controls they provide. The outcome of the analysis then informed a practitioner survey ( = 31) consisting of an online questionnaire followed by semi-structured interviews. A standard survey design methodology was followed [ 32, 33 ]. Data were collected via an online questionnaire aimed at project, program or portfolio managers from various industries, who had worked on complex projects over at least the past three years. Several channels were used to reach potential participants, including professional networks and LinkedIn. Survey participation was voluntary, with the data collection taking place mainly over four months in 2018.

In the remainder of this section we provide a thematic summary which combines all the key ndings. 4.1. Definition of complexity and volatility Complexity and volatility are widely acknowledged characteristics of today’s organisational problems, with broad recognition of their e ect on project success both in terms of product (scope and quality) and process (budget and schedule) outcomes. Yet, standard de nitions are lacking, with various authors using di erent nomenclatures and considering a wide range of characteristics under the umbrella of ‘complexity.’

From our analysis (see the summaries in Table 1), we have synthesised the following de nitions for our research: complexity relates to the presence of many interconnected parts; while volatility relates to the likelihood of rapid change.

Each can manifest itself along the following, not necessarily disjoint, dimensions: social, when related to people; technical, when related to technologies; and knowledge, when related to what is Uncertainty of goals, unclear meanings or stakeholders’ hidden agenda [ 42, 17 ] Complicated communication due to organisational or technical characteristics [ 43 ] Large number of technologies or interfaces involved [ 40, 17, 28, 41 ] Lack of knowledge at project start [ 7, 44 ] Novelty or uniqueness of the technical solution [ 44 ] Rate of change in the organisation [ 45 ] Rate of change in the technical solution [ 28, 45 ] Rate of change in the external environment, such as laws and regulations [ 46, 43 ] Time criticality of goals [ 11 ]

Social complexity or Knowledge complexity Social complexity or Technical complexity Technical complexity Knowledge complexity Knowledge complexity or Knowledge volatility or Technical volatility Social volatility or Knowledge volatility Technical volatility or Knowledge volatility Knowledge volatility Social volatility or Technical volatility known. In particular, the socio-technical systems which are the subject of organisational problem solving addressed by projects tend to combine multiple dimensions. 4.2. Complexity and volatility factors and how they a ect projects An assessment of complexity and volatility is considered bene cial in order to parametrise projects [ 47 ], however, there is no standard way to do so: in speci c industries, particularly software development and engineering (e.g., [ 48, 49, 50 ]), some authors have suggested sets of factors that could be used in such an assessment, but the picture remains patchy and it is unclear the extent these are actually used by practitioners. From our analysis of the literature, we have synthesised eleven factors which are su ciently general to apply across di erent types of project and industry, although no claim is made of their completeness. Through both literature analysis and practitioner interviews, we then provided a classi cation according to the complexity and volatility dimensions they relate to, and identi ed their potential impact on projects, which in turn may a ect either product or process success measures. Table 2 provides a summary, indicating the factors, their classi cation and potential impact. Note that some factors may relate to more than one dimension partly due to our subjective interpretation of the literature as well as that of the practitioners interviewed.

4.3. PM methodologies and their practices for risk control

Due to their characteristics, predictive and adaptive methodologies have long been seen as antithetic, each with its own “home ground” [ 51 ]: large, structurally complex systems and project teams in highly regulated industries with fairly stable requirements will use predictive methodologies; small systems and teams, in volatile environments and with readily available users and customers will use adaptive ones. However, the last decade has shown that complexity and volatility often combine in organisational problem solving, so that hybrid approaches have increased in practice [ 8, 9 ]. This was also con rmed by our practitioner survey.

It is important to stress, however, that external compliance requirements as well as organisational culture also in uence methodological choices, regardless of contextual characteristics. Predictive methodologies are often favoured, seen as providing better process predictability, maturity and repeatability, and not requiring senior management to relinquish central control in favour of power distribution and team self-organisation, typical of adaptive methodologies [ 51, 29 ]. A further factor is the readiness of an organisation, as adaptive methodologies rely heavily on stable, high performing teams and tacit knowledge [ 52 ].

These facts were con rmed by our survey participants, who indicated that, as a consequence, their current hybrid approaches are characterised by an overarching predictive structure with some adaptive components, the latter often related to the development of new digital technology or sub-systems.

4.4. Practices applied by practitioners for risk control

Our primary research on complexity and volatility allowed us a more nuanced assessment of predictive and adaptive practices to control risk.

We found that adaptive principles and practices are often challenged by social complexity, due to their expectations of key stakeholders’ continuous involvement in development and reviews, including their ability to agree common priorities early on, and their reliance on verbal communication and tacit knowledge within high performing teams. Instead, predictive approaches provide various well established practices to control risk from social complexity, particularly through stakeholder management, stringent governance and accountability, and explicit communication plans, to help overcome coordination and communication challenges.

Predictive methodologies also appear better equipped at dealing with social volatility than adaptive ones including social volatility within the organisation and the project team, while adaptive methodologies rely on stable high performing teams and tacit knowledge, predictive methodologies make use of change control and explicit documentation to deal with social volatility risk.

Adaptive approaches appear to perform better than predictive ones when it comes to both knowledge complexity and volatility, including the need to learn as one goes along, either due to the novelty or uniqueness of the technical solution or due to lack of su cient knowledge at start. Their lightweight processes made of fast and frequent cycles, with retrospective reviews and frequent stakeholder involvement and validation to learn lessons and make adjustments from one cycle to the next. This helps maintain problem solving alignment with changing needs and requirements, and develops a common understanding, clarify meaning and reduce uncertainty around goals, as stakeholders are required to agree priorities at each cycle, and to validate both assumptions and outcomes quickly and often. With a contained scope in each cycle, they are also e ective in dealing with technical volatility, reducing the risk of developing obsolete solutions.

Fast and frequent cycles driven by high performing teams also support the process of learning in the case of knowledge complexity or novel solutions, with the team quickly coming to terms with new or complex knowledge, while relying on verbal communication and tacit knowledge, and concentrating resources in each cycle speed up the process of delivery to time critical goals.

When it comes to technical complexity, risk control practices in predictive approaches include detailed up-front planning, minimising dependences between work packages and robust change control used to avoid scope creep. Moreover, formal risk management practices and the establishment of quality gates between phases, in which key stakeholders formally approve the deliverables of the previous phase, ensure that risk is not carried forward from one phase to the next. On the other hand, risk control practices in predictive approaches are much less speci c, relying mainly of standard decomposition into adaptive development cycles performed by a high performance team.

Dealing with uncertainty requires di erent strategies beyond traditional risk management, such as experimentation and the continuous generation of knowledge throughout the project lifecycle: in particular, fast iterations of learning may reduce areas of uncertainty and allow the project to react quickly to emerging issues, assuming the project is su ciently adaptive.

Some risk control practices were seen as methodologically neutral by practitioners: for instance, prototyping novel solutions can equally apply in predictive and adaptive approaches, as can the adoption of tried-and-tested solutions or establishing a single source of truth. Others are only meaningful in projects when a hybrid approach is assumed, like separating stable from variable elements of the project.

The emerging mapping from our research between factors, dimensions and predictive or adaptive risk control practices (’controls’ in the gure) is summarised in Figure 1.

5. Discussion and conclusion

We have investigated how to match characteristics of organisational problems to speci c PM practices, in order to minimise project risk and maximise project success. By analysing complexity and volatility into their prevalent dimensions and manifestations, as displayed in Table 2, and PM methodologies into their constituent controls, as displayed in Figure 1, we were able to investigate a ne-grain mapping between speci c risk factors and methodological controls, both from a theoretical standpoint and in conversation with practitioners. The identi cation of social, technical and knowledge dimensions is important as it allows us to better tailor risk controls, including consideration of trade-o s at the interaction between the social and the technical.

This mapping has also some limitations, both due to the level of subjective interpretation involved, and the relatively small sample of practitioners taking part in the study, although triangulation between primary and secondary evidence has provided some mitigation against the latter. Nevertheless, we claim that this mapping provides an empirical basis for a rst attempt at de ning a PM hybridisation framework based on organisational problem characteristics. This is the focus of our ongoing research: brie y, at its essence is a POE characterisation of both socio-technical organisational problems, with their complexity and volatility dimensions, and of projects as problem solving processes, with the mapping in Figure 1 applied at each step of the process to inform how risk arising from complexity and volatility should be best controlled, with trade-o s assessed against project objectives and validated by stakeholders through the POE process pattern.

Acknowledgments References

Thanks go to all survey participants, and to Osram and Webasto for support and access to case studies. [1] F. Carton, F. Adam, D. Sammon, Project management: a case study of a successful ERP implementation, International Journal of Managing Projects in Business 1 (2008) 106–124. [2] Project Management Institute, A Guide to the Project Management Body of Knowledge (PMBOK®Guide), Project Management Institute, Incorporated, 2013.

References

[3] Project Management Institute , Success in disruptive times, Technical Report, Pulse of the Profession, Project Management Institute (PMI) , 2018 .

[4] Project Management Institute , A Guide to the Project Management Body of Knowledge (PMBOK®Guide) , Project Management Institute, Incorporated , 2017 .

[5] R. Joslin , R. Müller , Relationships between a project management methodology and project success in di erent project governance contexts , International Journal of Project Management 33 ( 2015 ) 1377 - 1392 .

[6] Project Management Institute , Ahead of the curve , Technical Report, Pulse of the Profession, Project Management Institute (PMI) , 2018 .

[7] T. Ahern , B. Leavy , P. Byrne , Complex project management as complex problem solving: A distributed knowledge management perspective , International Journal of Project Management 32 ( 2014 ) 1371 - 1381 .

[8] G. Theocharis , M. Kuhrmann , J. Münch , P. Diebold, Is water-scrum-fall reality? on the use of agile and traditional development practices , in: International Conference on Product-Focused Software Process Improvement , Springer, 2015 , pp. 149 - 166 .

[9] K. Kuusinen , P. Gregory , H. Sharp , L. Barroca, Strategies for doing agile in a non-agile environment , in: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement , ACM, 2016 , p. 5 .

[10] L. R. Vijayasarathy , C. W. Butler , Choice of software development methodologies: Do organizational, project, and team characteristics matter? , IEEE Software 33 ( 2016 ) 86 - 94 .

[11] J. Geraldi , H. Maylor , T. Williams , Now, let's make it really complex (complicated) a systematic review of the complexities of projects , International Journal of Operations & Production Management 31 ( 2011 ) 966 - 990 .

[12] J. G. Hall , L. Rapanotti , A design theory for software engineering , Information and Software Technology 87 ( 2017 ) 46 - 61 .

[13] M. O'Halloran , J. G. Hall , L. Rapanotti, Safety engineering with COTS components , Reliability Engineering & System Safety 160 ( 2017 ) 54 - 66 .

[14] A. Nkwocha , J. G. Hall , L. Rapanotti , Design rationale capture for process improvement in the globalised enterprise: an industrial study , Software & Systems Modeling 12 ( 2013 ) 825 - 845 .

[15] J. G. Hall , L. Rapanotti, Problem frames for socio-technical systems , in: P. Zaphiris, C. S. Ang (Eds.), Human Computer Interaction: Concepts , Methodologies, Tools, and Applications , Information Science Reference, 2009 , pp. 713 - 731 .

[16] S. Sarker , S. Chatterjee , X. Xiao , A. Elbanna , The sociotechnical axis of cohesion for the is discipline: Its historical legacy and its continued relevance , Mis Quarterly 43 ( 2019 ) 695 - 720 .

[17] T. M. Williams , The need for new paradigms for complex projects , International journal of project management 17 ( 1999 ) 269 - 273 .

[18] R. Atkinson , Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria , International journal of project management 17 ( 1999 ) 337 - 342 .

[19] S. R. Nidumolu , Standardization, requirements uncertainty and software project performance , Information & Management 31 ( 1996 ) 135 - 150 .

[20] L. Wallace , M. Keil , Software project risks and their e ect on outcomes , Communications of the ACM 47 ( 2004 ) 68 - 73 .

[21] H. Barki , S. Rivard , J. Talbot , An integrative contingency model of software project risk management , Journal of management information systems 17 ( 2001 ) 37 - 69 .

[22] Project Management Institute , Success Rates Rise Transforming the high cost of low performance , Technical Report, Pulse of the Profession, Project Management Institute (PMI) , 2017 .

[23] F. Böhle , E. Heidling, Y. Schoper , A new orientation to deal with uncertainty in projects , International Journal of Project Management ( 2015 ).

[24] G. Purdy , Iso 31000 : 2009 - setting a new standard for risk management , Risk Analysis: An International Journal 30 ( 2010 ) 881 - 886 .

[25] M. Fan , N.-P. Lin , C. Sheu , Choosing a project risk-handling strategy: An analytical model , International Journal of Production Economics 112 ( 2008 ) 700 - 713 .

[26] T. Aven , On the new iso guide on risk management terminology , Reliability engineering & System safety 96 ( 2011 ) 719 - 726 .

[27] D. Cleden , Managing project uncertainty , Routledge , 2017 .

[28] J. Conklin , Dialogue mapping: Building shared understanding of wicked problems , Wiley, 2006 .

[29] D. West , M. Gilpin , T. Grant , A. Anderson , Water-scrum-fall is the reality of agile for most organizations today , Forrester Research 26 ( 2011 ).

[30] E. Mumford, The story of socio-technical design: Re ections on its successes, failures and potential , Information systems journal 16 ( 2006 ) 317 - 342 .

[31] R. H. von Alan , S. T. March , J. Park, S. Ram, Design science in information systems research , MIS Quarterly 28 ( 2004 ) 75 - 105 .

[32] B. J. Oates , Researching information systems and computing, Sage , 2005 .

[33] F. J. Fowler Jr , Survey research methods, Sage publications, 2013 .

[34] M. Bosch-Rekveldt , Y. Jongkind , H. Mooi , H. Bakker , A. Verbraeck , Grasping project complexity in large engineering projects: The toe (technical, organizational and environmental) framework , International Journal of Project Management 29 ( 2011 ) 728 - 739 .

[35] D. Baccarini , The concept of project complexity a review , International journal of project management 14 ( 1996 ) 201 - 204 .

[36] A. Tiwana , M. Keil , The one-minute risk assessment tool , Communications of the ACM 47 ( 2004 ) 73 - 77 .

[37] C. Sauer , A. Gemino , B. H. Reich , The impact of size and volatility on it project performance , Communications of the ACM 50 ( 2007 ) 79 - 84 .

[38] N. Nurmuliani , D. Zowghi , S. Powell , Analysis of requirements volatility during software development life cycle , in: Software Engineering Conference, 2004 . Proceedings. 2004 Australian, IEEE, 2004 , pp. 28 - 37 .

[39] M. Singh , R. Vyas , Requirements volatility in software development process , International Journal of Soft Computing 2 ( 2012 ) 2012 .

[40] D. Baccarini , Management of risks in information technology projects , Industrial Management & Data Systems 104 ( 2004 ) 286 - 295 . doi: 10 .1108/02635570410530702.

[41] G. Girmscheid , C. Brockmann , The inherent complexity of large scale engineering projects , Project perspectives 29 ( 2008 ) 22 - 26 .

[42] M. Saynisch , Mastering complexity and changes in projects, economy, and society via project management second order (pm-2 ), Project Management Journal 41 ( 2010 ) 4 - 20 .

[43] Y. Lu , L. Luo , H. Wang , Y. Le , Q. Shi , Measurement model of project complexity for large-scale projects from task and organization perspective , International Journal of Project Management 33 ( 2015 ) 610 - 622 .

[44] J. G. Geraldi , G. Adlbrecht, On faith, fact, and interaction in projects , Project Management Journal 38 ( 2007 ) 32 - 43 .

[45] S. J. Whitty , H. Maylor , And then came complex project management (revised ), International Journal of Project Management 27 ( 2009 ) 304 - 310 .

[46] Q. He , L. Luo , Y. Hu , A. P. Chan , Measuring the complexity of mega construction projects in china, a fuzzy analytic network process analysis , International Journal of Project Management 33 ( 2015 ) 549 - 563 .

[47] K. Remington , R. Zolin , R. Turner , A model of project complexity: distinguishing dimensions of complexity from severity , in: Proceedings of the 9th International Research Network of Project Management Conference , volume 11 , IRNOP Berlin, 2009 .

[48] P. Fitsilis , Measuring the complexity of software projects , in: 2009 WRI World Congress on Computer Science and Information Engineering , volume 7 , IEEE, 2009 , pp. 644 - 648 .

[49] P. Clarke , R. V. O'Connor, The situational factors that a ect the software development process: Towards a comprehensive reference framework , Information and Software Technology 54 ( 2012 ) 433 - 447 .

[50] G. Kalus , M. Kuhrmann , Criteria for software process tailoring: a systematic review , in: Proceedings of the 2013 International Conference on Software and System Process , ACM , 2013 , pp. 171 - 180 .

[51] B. Boehm , R. Turner , Balancing agility and discipline: Evaluating and integrating agile and plandriven methods , in: Software Engineering , 2004 . ICSE 2004. Proceedings. 26th International Conference on, IEEE , 2004 , pp. 718 - 719 .

[52] I. Bider , Analysis of agile software development from the knowledge transformation perspective , in: International Conference on Business Informatics Research , Springer, 2014 , pp. 143 - 157 .