=Paper=
{{Paper
|id=Vol-1854/Paper7
|storemode=property
|title=Problem Drift: a Risk Model for Complex Socio-technical Projects
|pdfUrl=https://ceur-ws.org/Vol-1854/Paper7.pdf
|volume=Vol-1854
|authors=Silvana Costantini,Jon Hall,Lucia Rapanotti
|dblpUrl=https://dblp.org/rec/conf/caise/CostantiniHR17
}}
==Problem Drift: a Risk Model for Complex Socio-technical Projects==
https://ceur-ws.org/Vol-1854/Paper7.pdf
Proceedings of STPIS'17
Problem drift: a risk model for complex
socio-technical projects
Silvana Costantini, Jon G. Hall, and Lucia Rapanotti
The Open University, UK
Abstract. Increased globalisation and market volatility put pressure on
organisations to become more flexible and explore new technologies and
operational domains, so that projects are becoming more complex, with
increasing uncertainty and risk. Complex project management is chal-
lenging for both traditional and agile approaches: traditional techniques
may be left behind by unrecognised environmental change in a volatile
context, while in a stable environment, agile may require too expensive
interaction with the client. One area of growing interest is how tradi-
tional and agile may combine to increase the chance of project success.
Yet how to strike a balance between the two remains poorly understood.
In this position paper, we propose a model for complex socio-technical
projects related to risk arising from volatility, that tries to explain the
balance of agile and traditional. The model introduces the concept of
drift rate as a measure of volatility, with the impact of drift related to
loss of development resource, risk accumulation as a function of resource
expenditure and drift rate, and validation the means to manage that
risk.
1 Introduction
Organisations must adapt to their changing business environment to remain
profitable, competitive and able to meet their strategic goals. In large organisa-
tions, projects are the instrument of choice for such adaptations, each project
addressing specific organisational socio-technical problems. Increased globalisa-
tion and market volatility are putting pressure on organisations to become more
flexible and explore new technologies and operational domains, so that projects
are becoming more complex, with increasing uncertainty and risk.
There are many sources of complexity and uncertainty. From a technical per-
spective, they stem from the combination, interactions and inter-dependencies
of technologies and the pace of technical change [5]; from a social perspective,
from “the number and diversity of stakeholders in the social network around a
project” [5], their culture and power relations; and, from a knowledge perspec-
tive, the embedded, distributed and tacit nature of knowledge [1] combined with
inherent ‘wicked’ [13] features prevents complex problems from being entirely
specifiable upfront. Moreover, the volatility of the external environment, a key
driver for organisational change, accelerates and magnifies complexity and un-
certainty [1]. All combine to increase risk and failure associated with projects
[4].
Edited by S. Kowalski, P. Bednar and I. Bider 47
Proceedings of STPIS'17
At a fundamental level, all project management approaches, whether tra-
ditional (‘plan-driven’) or ‘agile,’ seek to control uncertainty and manage risk,
while making best use of resources. Complex projects are problematic for both
approaches: environment volatility is a challenge for plan-driven ones, technical
complexity for agile ones, and knowledge complexity for both. While often con-
sidered as antithetic, plan-driven and agile practices are increasingly combined
in practice, particularly in software development, in an ad hoc manner to reap
their benefits, with various degrees of success [10]: the reality is that in complex
projects deviations and failure rates remain high [3].
[1] advocate that complex project management should be seen as a form of
complex problem solving. This is also our position, therefore we apply a complex
problem solving framework to explicate and model features of traditional and
agile approaches. The framework is particularly strong at process modelling [7],
and this allows us to consider their potentially constructive combination, indexed
by organisational and problem characteristics.
The ultimate aim of the work is to equip organisations with the tools to
understand better the nature of their problems and tailor their project manage-
ment practices to their needs and culture. This position paper is a step towards
this aim: it seeks to explicate risk characteristics of project management ap-
proaches when interpreted as problem solving processes, through the lens of
a design theoretic framework for complex problem solving, called Problem Ori-
ented Engineering [8]. Specifically, we propose a model to explicate how different
approaches contribute to risk accumulation and mitigation.
2 Risk and uncertainty
The Project Management Institute’s PMBOK defines risk as “an uncertain event
or condition that, if it occurs, has a positive or negative effect on one or more
project objectives such as scope, schedule, cost, or quality” [12], and many risk
management approaches exist, aimed at identifing, quantifying and mitigating
risk. The notion of ‘uncertain event’ is an indication that uncertainty and risk
are closely related concepts, although there is growing recognition of their dif-
ferences. For instance, [4] takes a knowledge-centric view to distinguish between
different kinds of uncertainty, summarised in the four quadrant model repro-
duced in Figure 1.
In Cleden’s view [4], all projects start with inherent uncertainty, some of
which is susceptible to risk analysis, which transforms some uncertain events into
risk, but still leaves latent uncertainly, which requires management approaches
distinct from traditional risk management.
One of our objectives is to explicate how traditional and agile project man-
agement deal with risk and uncertainty in complex projects.
©Copyright held by the author(s) 48
Proceedings of STPIS'17
Fig. 1. Four quadrant model (reproduced from [4]).
3 Complex project management as complex problem
solving
Ahern et al. [1] argue that complex project management should be seen as a form
of complex problem solving. They highlight two fundamental aspects: learning,
which allows the creation, organisation and coordination of emergent knowledge
that cannot be specified at the outset; and fostering a common will of mutual
interest among project stakeholders. The latter is similar to Conklin’s coher-
ence [5] — stakeholders having shared understanding and shared commitment
to project objectives and success. These two aspects relate directly to knowledge
and social complexity.
In line with this thinking, in our research, we look at complex project man-
agement through the lens of Problem Oriented Engineering (POE, see [8] for a
comprehensive introduction), a design theoretic framework for complex problem
solving. POE has been applied in real-world case studies as a systematic ap-
proach to address a wide range of engineering and organisational problems (for
instance, [6, 9]).
POE is concerned both with different types of knowledge, spanning problem
and solution spaces, and with stakeholders with a role in the problem solving
process. In essence, a POE problem is a recognised (real-world) need in con-
text, whose solution satisfies it. Arriving at a solution is a process in which
need, context and solution are progressively ‘explored’ and ‘validated.’ During
exploration, learning takes place so that stakeholders come together to create,
organise and exchange knowledge, which reduces knowledge complexity, while
validation records stakeholders’ satisfaction on the outcome of exploration, re-
ducing uncertainty and contributing to coherence.
Edited by S. Kowalski, P. Bednar and I. Bider 49
Proceedings of STPIS'17
Explorations take time and effort, so that expenditure and project risk is
accumulated during such activities. Through validation, explorers transfer risk
to validating stakeholders. When validation fails, backtracking to a previous
state, including project start, may be the outcome.
In its idealised form, such a process is captured by the pattern in Figure 2.
The pattern relates fundamental activities and actors and is not a prescriptive
model — a process may be linear, iterative, spiral, fractal, etc.
Validation
Problem exploration Solution exploration Validation
(problem explorers) (solution explorers)
Fig. 2. POE Process Pattern (PPP): stakeholders interacting in problem solving ac-
tivities — problem exploration and solution exploration are interleaved with validation
(adapted from [7]).
In summary, as caputured by the PPP, complex project management is a
problem solving endeavour to design solutions to organisational problems (need
in context), and in which knowledge and social complexity are tackled through
problem and solution explorations and validations, reducing uncertainty and
mitigating risk.
Our conjecture is that PPP and its underlying concepts provide a fruitful
theoretical framework in which to analyse risk characteristics of traditional and
agile project management approaches when interpreted as problem solving pro-
cesses. In the next section we put this idea to the test by proposing a model for
project risk arising from problem (need in context) volatility.
4 Problem drift
Traditional project management approaches expect projects to be completely
planned in advance, with their execution merely a process of implementation.
As a consequence, resources are committed to a chosen solution early in the
project lifecycle. Even if we were to discount knowledge complexity — which
prevents both problem and solution from being completely knowable at the onset
— such a commitment is particularly risky in the presence of problem volatility,
due, for example, to rapid market or technological change (context volatility)
or stakeholder conflicts (need volatility). If one were to take the same approach
to archery, one would identify a target and aim, close ones eyes, pull back the
string and let an arrow go. In the case that the target is static, such an approach
might be appropriate. But what in the case of a moving target? The arrow will
land where the original target was, not where it has moved to.
Thus, in traditional approaches, fixing a solution choice early incurs the risk
that the problem will have moved as a product of the volatility of the organi-
sation’s problems (need in context): the more volatile the problem, the higher
©Copyright held by the author(s) 50
Proceedings of STPIS'17
the likelihood that a delivered solution will ‘miss’ its target, i.e., the delivered
solution does not satisfy the need in context. Instead, we must track the target
as it ‘moves’: this is where validation has a role to play.
To understand ‘problem drift’, we propose the ‘problem drift model’ illus-
trated in Figure 3:
– the vertical axis represents ‘problem drift’, which is a measure of divergence
from the problem the project was set up to solve and that which is in the
real world, as a result of problem volatility;
– the horizontal axis represents resource use (equivalently, the passage of time)
from most recent validation (initially project front-end);
– we assume an ‘acceptable drift range’ (the shaded area), where the divergence
is not sufficient to invalidate development of the solution;
– the ‘drift rates’ lines indicate the degree of problem volatility: from high (V1),
i.e., the environment changes very quickly, to low (V3), where it changes more
slowly.
Marked on the horizontal axis is a ‘validation point,’ which is the point where
validation against the problem is sought on some validation artefact from stake-
holders. Three situations are considered: i) in a situation of low drift (V3), drift
has occurred but the validation artefact can still be validated; ii) when medium
drift (V2), drift leads to a marginal call on the boundary between acceptable and
unacceptable drift; iii) when high drift (V1), the validation artefact cannot be
validated and some or all development resource is lost: based on the knowledge
gained, a previous project state may be revisited or, in the extreme, the project
is abandoned.
unacceptable drift
ater
ift
dr
igh
Drift
e
t rat
:H
drif
dium
V1
me
V2:
marginal drift
ft rate
V3: Low dri
acceptable drift
Acceptable drift region
Resource use
vali
dati
on
poi
nt
Fig. 3. Problem drift model
Under this model, the impact of problem drift relates to lost development
resource, and risk accumulation is then a function of resource expenditure and
drift rate, with validation the means to manage risk. This is consistent with
Edited by S. Kowalski, P. Bednar and I. Bider 51
Proceedings of STPIS'17
empirical evidence [15] in software projects which suggests that more frequent
developer/customer communication (a form of validation) lessens the impact of
requirements volatility.
Seen this way, we may interpret the role of validation in various project
management approaches. At the extremes:
– in extreme plan-driven methods, such as Royce’s posited Waterfall model
[14], validation would occur only at project end. Unless the drift rate is
zero or extremely low, any delivered solution to the original problem will be
unacceptable. This corresponds to traditional approaches being appropriate
only in situations of low drift (with the caveat that knowledge complexity
would still pose a challenge);
– in extreme agile methods, validation occurs early and often so that, even in
situations of medium to high drift, unacceptable drift is never experienced.
The related risk of failure is thus reduced.
One might ask, if agile copes with all drift rates, why not eschew plan-driven
methods altogether? Indeed, this view might be the source of the popular view
that all processes should be agile. Our model also explains why this is not the
case, however: validation requires time and effort, and so consumes resources,
diverting them from development. As a result, agile processes ‘pay’ for their
management of risk through potentially expensive validation.
There is therefore a balance to be struck between agile and plan-driven: the
model suggests that the optimal balance is to validate at, or just before, the
point of marginal drift (V2). To do this a number of things are required: i)
the measurement of problem volatility; ii) the notion of acceptable drift; and iii)
ways of combining traditional and agile methods to support the optimal balance.
5 Discussion and Future Work
Complex project management is challenging for both traditional and agile ap-
proaches: traditional techniques may be left behind by unrecognised environ-
mental change in a volatile context, while in a stable environment, agile may
require too expensive interaction with the client. One area of growing interest is
how traditional and agile may combine to increase the chance of project success.
Yet how to strike a balance between the two remains poorly understood.
In this position paper, we have proposed a model for complex socio-technical
projects related to risk arising from problem volatility, that tries to explain
the balance of agile and traditional. The model is located within the Problem
Oriented Engineering (POE) framework of the second and third authors [8]
and, in particular, derives from the POE Process Pattern (PPP). The model
introduces the concept of drift rate as a measure of problem volatility, with the
impact of problem drift related to loss of development resource, risk accumulation
as a function of resource expenditure and drift rate, and validation the means
to manage risk.
©Copyright held by the author(s) 52
Proceedings of STPIS'17
A different characterisation of tradition versus agile is given in [2], which takes
inspiration from the SECI model for knowledge transformation in organisations
[11] to explicate and compare how tacit, explicit and embedded knowledge are
transferred during traditional vs. agile software development. The main insight is
that traditional methods rely on explicit forms (e.g., requirements specifications,
design models) to communicate with a variety of specialist stakeholders, while
in agile methods, tacit knowledge is shared through ‘Socialization’ (i.e., direct
communication between problem owners and development team) supported by
‘Embedment’, in which knowledge is embedded in the software system through
coding. With reference to the PPP, these forms of knowledge transfer fall within
exploration, but the different types of transfer are not differentiated by the pat-
tern. Instead, PPP is concerned with the accumulation of risk during exploration
and its mitigation via validation. This suggests that the two approaches can be
combined and may thus deliver orthogonal benefits.
The drift model proposed presupposes that drift can be measured and pre-
dicted. The objective of future research will be to explore if these measurement
and prediction are feasible and how to implement them. For instance, it is plau-
sible that measuring how problem descriptions change over time is one possible
proxy measure for context volatility. However, more sophisticated notions should
also be considered.
Supposing that our drift model works, the question arises as to the extent we
can optimally combine agile with traditional processes. This would require some
hybrid process architecture to be developed. The second and third authors have
already taken preliminary steps in the analysis of the agile/plan-driven mix in
the context of software projects in [7], but much remains to be done.
Finally, in our model we have focused on problem volatility and process activ-
ity, irrespective of organisational culture, which is an acknowledged key success
factor for agility [2]. Any cultural implication of our model will be considered in
future research.
Bibliography
[1] Terence Ahern, Brian Leavy, and PJ Byrne. Complex project management as
complex problem solving: A distributed knowledge management perspective. In-
ternational Journal of Project Management, 32(8):1371–1381, 2014.
[2] Ilia Bider. Analysis of agile software development from the knowledge transforma-
tion perspective. In International Conference on Business Informatics Research,
pages 143–157. Springer, 2014.
[3] Fritz Böhle, Eckhard Heidling, and Yvonne Schoper. A new orientation to deal
with uncertainty in projects. International Journal of Project Management, 2015.
ISSN 0263-7863.
[4] David Cleden. Managing project uncertainty. Gower Publishing, Ltd., 2012.
[5] Jeffrey Conklin. Dialogue mapping: Building shared understanding of wicked prob-
lems. Wiley, 2006.
[6] Jon G. Hall and Lucia Rapanotti. Assurance-driven design in problem oriented
engineering. International Journal on Advances in Systems and Measurements, 2
(1):119–130, 2009.
Edited by S. Kowalski, P. Bednar and I. Bider 53
Proceedings of STPIS'17
[7] Jon G. Hall and Lucia Rapanotti. Towards a design-theoretic characterisation
of software development process models. In Proceedings of the Fourth SEMAT
Workshop on General Theory of Software Engineering, pages 3–14, 2015.
[8] Jon G Hall and Lucia Rapanotti. A design theory for software engineering. In-
formation and Software Technology, 87:46–61, 2017.
[9] Jon G. Hall, Lucia Rapanotti, and Michael A. Jackson. Problem oriented soft-
ware engineering: Solving the package router control problem. IEEE Transactions
on Software Engineering, 34(2):226–241, 2008. ISSN 0098-5589. doi: 10.1109/
TSE.2007.70769. URL http://ieeexplore.ieee.org.libezproxy.open.ac.uk/
ielx5/32/4476751/04384506.pdf?tp=&arnumber=4384506&isnumber=4476751.
[10] Kati Kuusinen, Peggy Gregory, Helen Sharp, and Leonor Barroca. Strategies for
doing agile in a non-agile environment. In Proceedings of the 10th ACM/IEEE
International Symposium on Empirical Software Engineering and Measurement,
page 5. ACM, 2016.
[11] Ikujiro Nonaka. A dynamic theory of organizational knowledge creation. Organi-
zation science, 5(1):14–37, 1994.
[12] Project Management Institute. A Guide to the Project Management Body of
Knowledge (PMBOK R Guide). Project Management Institute, Incorporated,
2013.
[13] Horst W.J. Rittel and Melvin M. Webber. Dilemmas in a general theory of plan-
ning. Policy sciences, 4(2):155–169, 1973.
[14] Winston W Royce et al. Managing the development of large software systems. In
proceedings of IEEE WESCON, volume 26, pages 1–9. Los Angeles, 1970.
[15] Didar Zowghi and Nur Nurmuliani. A study of the impact of requirements volatil-
ity on software project performance. In Software Engineering Conference, 2002.
Ninth Asia-Pacific, pages 3–11. IEEE, 2002.
©Copyright held by the author(s) 54