=Paper=
{{Paper
|id=None
|storemode=property
|title=Towards an Economic Foundation for the Decision between Agile and Plan-driven Project Management in a Business Intelligence Context
|pdfUrl=https://ceur-ws.org/Vol-1048/paper4.pdf
|volume=Vol-1048
|dblpUrl=https://dblp.org/rec/conf/wsbi/OtyepkaMM12
}}
==Towards an Economic Foundation for the Decision between Agile and Plan-driven Project Management in a Business Intelligence Context==
https://ceur-ws.org/Vol-1048/paper4.pdf
Towards an Economic Foundation for the Decision
between Agile and Plan-driven Project Management in a
Business Intelligence Context
Sarah Otyepka, Benjamin Mosig, Marco C. Meier
FIM Research Center
University of Augsburg
Abstract
Lacking a formal yet practical decision model, nowadays decision makers mostly follow
corporate guidelines or their intuition when it comes to the decision between agile and
plan-driven project management in Business Intelligence projects. As one size does not fit
all, using management methods hyped by temporary fashion or other management meth-
ods not adapted to the situation bears the risk of project failure. Thus, this paper propos-
es a risk-adjusted net present value-based model to support decision makers in the selec-
tion of the appropriate management method for Business Intelligence projects. We focus
on two decisive risk parameters – the likelihood of environmental changes and the peril
of improper system integration – and a project’s estimated cash flows. As a result, the
tradeoff between different characteristics of risks and cash flows in a specific project is
formalized and made transparent. In summary, this research-in-progress paper sketches
the idea of a practical decision model that improves the foundation for the selection of the
appropriate management method.
1 Motivation
Although studies suggest that non-technical factors such as project management dominate
a Business Intelligence (BI) project’s success (Adamala & Cidrin, 2011), the decision on
a project management method in BI projects, such as Scrum, Extreme Programming, the
waterfall model or Sandboxing, is often not based on rational considerations (Project
Management Methodologies. n. d.). Examples include German authorities that require
any in-house information technology (IT) project to use the V-model XT (Höhn &
Höppner, 2008, p. IX). In other cases, the decision on the project management method is
the result of nothing more than “this is what we’ve always done” (Project Management
Sarah Otyepka, Benjamin Mosig, Marco C. Meier
Methodologies. n. d.). Thus, many companies or authorities seem to give either mandato-
ry “one-size-fits-all” instructions or apply project management methods inconsiderately.
The Project Management Institute defines project management as “the application of
knowledge, skills, tools and techniques to project activities to meet the project require-
ments” (Project Management Institute, 2008, p. 6). While project management describes
the necessity to use knowledge, skills and tools, a project management method(ology)
describes more precisely how this knowledge, skills and tools look like: “A methodology
is a set of guidelines or principles that can be tailored and applied to a specific situation.
In a project environment, these guidelines might be a list of things to do” (Charvat, 2003,
p. 3). As the terms “method” and “methodology” are often used interchangeably in IT-
related literature (Boehm & Turner, 2003; Charvat, 2003; Larman & Basili, 2003; Moss,
2009), we will consistently use “method” in the following.
There are important differences between BI and other IT or software development pro-
jects that need to be addressed. While BI projects are rather data-centric business integra-
tion projects, being closely linked to a company’s strategy, software development projects
are rather code-centric (Moss, 2009). This is the reason why non-technical success factors
dominate technical success factors in BI projects (Adamala & Cidrin, 2011). Hence, the
selection of an appropriate project management method is critical. Instead of an ex ante
determined project management method that might ultimately result in project failure, a
situation-based decision prevents the application of a potentially inappropriate one (Char-
vat, 2003, pp. 18-20). Therefore, a situation-based decision seems more promising. BI
projects – as any kind of projects – tie up capital and accordingly should be seen as in-
vestments supposed to increase shareholder value. For the required evaluation of suitable
project management methods, a value-based and future-oriented approach is adequate
(Coenenberg, Mattner, & Schultze, 2003, p. 3; Mertens, 1999, p. 11).
Surprisingly, “there are few [researchers] that compare agile method projects with those
using traditional approaches, which one would expect when a new range of methods […]
claims to be superior” (Conboy, 2009, p. 331). Furthermore, decisions on project man-
agement methods lack a fact-based and comparable approach considering significant
parameters and, therefore, bear the risk of incorrect decisions. After an initial literature
review (including: Black, Boca, Bowen, Gorman, & Hinchey, 2009; Boehm & Turner,
2003; Charvat, 2003; Conboy, 2009; Fowler & Highsmith, 2001; Kerzner, 2009; Moss &
Atre, 2003; Wysocki, 2011), there seems to be no approach of a quantitative decision
model for the selection of the most appropriate project management method in a specific
BI project setting. This leads us to following research question: “What would be an ap-
propriate value-based decision model that supports decision makers in their selection of a
specific project management method for a given BI project?”
WSBI 2012 – Agile and Plan-driven Project Management in a Business Intelligence Context 39
This research-in-progress paper first identifies requirements for a practical decision mod-
el and provides necessary theoretical background. The subsequent section sketches the
idea of a value-based decision model using a simplified example for demonstration. The
model enables the determination of the most appropriate project management method in a
particular BI project setting. The last section discusses limitations of the current model
sketch and describes the further research agenda to enhance both maturity and generali-
zability of the decision model.
2 Theoretical Background
As indicated above, the decision on an appropriate project management method in a BI
context lacks a comprehensible and practical decision model. From a scientific point of
view, we expect a decision model to fulfill the following requirements: It has to (1) be
applicable to a class of problems, (2) contribute new findings to the body of knowledge,
(3) be comprehensible and reproducible, and (4) generate value for a user now or in fu-
ture (Österle et al., 2010, p. 666). Furthermore, we additionally expect the model to (5) be
economically feasible to ensure its applicability, i.e., the effort of application should be
justified by its benefits, and (6) adopt a value-based approach using quantitative metrics
as indicated in section 1 (Coenenberg et al., 2003, p. 3; Mertens, 1999, p. 11).
As object of research, we limit the considered project management methods in a first step
to two extremes: a purely agile project management method (A) and a purely plan-driven
project management method (P). Regarding the former, we need to distinguish agile pro-
ject management methods from iterative and incremental approaches. While iterative
development is “a rework scheduling strategy in which time is set aside to revise and
improve parts of the system”, incremental development is “a staging and scheduling strat-
egy in which various parts of the system are developed at different times or rates and
integrated as they are completed” (Cockburn, 2008, p. 27). A does not necessarily include
iterative or incremental elements, although changes during project runtime might suggest
including one or both of the two approaches – corresponding with the principle to “value
responding to change over following a plan” (Fowler & Highsmith, 2001, p. 29).
Both extremes, A and P, can be realized differently, for example with Scrum or Extreme
Programming in case of A and CMMI (Capability Maturity Model Integration) or other
plan-driven project management methods such as the waterfall method in case of P
(Boehm & Turner, 2003, p. 14 ff.). Since we refer to the major characteristics of A and P,
the key findings of this paper can be applied to any realized A or P. P promises planning
reliability, strong documentation, comparability and repeatability (Boehm & Turner,
2003). However, the major shortcoming associated with P has been widely criticized: It is
Sarah Otyepka, Benjamin Mosig, Marco C. Meier
inappropriate to handle rapidly changing environments and customer requirements
(Abrahamsson, Conboy, & Wang, 2009, p. 281). Consequently, dynamism is the major
risk to be addressed in projects applying P (Maruping, Venkatesh, & Agarwal, 2009, p.
377). On the other hand, A is characterized as lightweight processes with close customer
collaboration and short iterative cycles resulting in independently running modules
(Boehm & Turner, 2003, p. 17; Fowler & Highsmith, 2001). The trend to value working
software over documentation, individuals and interactions over processes and tools and
responding to change over following a plan bear the risk of delivering perfectly working
individual modules which are not thoroughly working as an integrated system (Black et
al., 2009, p. 39; Fowler & Highsmith, 2001, p. 29). Therefore, the risk of improper inte-
gration is the major risk to be addressed when applying A. In summary, environmental
dynamism and improper system integration are the two major risks to be taken into ac-
count in the proposed basic decision model.
Literature shows that there are few researchers who compare projects applying one pro-
ject management method or the other (Conboy, 2009, p. 331). Therefore, we seek for
support on the decision between A and P in a specific project setting. We are aware that
not all relevant project management methods can be classified into one of the two ex-
tremes. However, to identify key issues in a first step, we will not take into account those
as well as the possibility of merging two project management methods – like already
proposed, for example, in Boehm and Turner’s five step risk-based approach (Boehm &
Turner, 2003). While their approach evaluates the project’s characteristics in detail, they
do not take into account the project’s expected cash flows, and hence lack the desired
quantitative nature.
Since there is no decision model that meets the aforementioned requirements, we suggest
a value-based and future-oriented Net Present Value (NPV) model taking into account the
project’s planned cash flows and integrating the risk of dynamism and the risk of improp-
er system integration.
3 Proposition of a Decision Model and Demonstration Example
The example project has the objective to set up three reports (R1, R2, and R3) based on a
data warehouse (DW) that integrates data from three operational databases (ODB1,
ODB2, and ODB3) (see Figure 1). The project has to be finished within the next nine
months.
WSBI 2012 – Agile and Plan-driven Project Management in a Business Intelligence Context 41
R1 R2 R3
Data analysis,
distribution
and presentation
€ -10,000 € -30,000 € -40,000
€ +5,000 € +2,500 € +1,500
Data integration DW € -60,000
and storage Legend
R = report
DW = data warehouse
€ -20,000 € -20,000 € -20,000 ODB = operational database
Data transfer One-time cash flow
and retransfer
ODB 1 ODB 2 ODB 3 Recurring cash flow
Figure 1: Target architecture including component-based cash flows
The three reports provide different value to the company. Their implementation costs
depend on the reports’ complexity. The most important report R1 fulfills new mandatory
regulatory requirements and, by preventing sanctions, provides the highest value. It re-
quires the integration of two operational databases (ODB1 and ODB2). R2, a better CRM
(Customer Relationship Management) report, has the second priority and also requires
ODB1 and ODB2. R3 shows the comparatively smallest value contribution. It offers
more detailed analyses for controlling purposes and requires all three source systems to
be available.
Costs and returns are supposed to be based on expert’s estimation. For the simplified
example we assume following one-time costs that are identical irrespective of the selected
project management method: Developing R1 causes costs of €10,000, R2 €30,000, and
R3 €40,000. Once the reports are in place, they generate recurring returns of €5,000 (R1),
€2,500 (R2), and €1,500 (R3) in each period. A period thereby equals a quarter (three
months). Integrating an ODB causes costs of €20,000 each. The development of the DW
costs €60,000, which will be – in case of applying A – assigned mostly to the first peri-
od t1, taking into account that the data model has to be defined at the beginning. The costs
of the remaining modules are assigned to the period when they are implemented and test-
ed (see Figure 2). In case of applying P, we assume that the costs in period t1 are slightly
higher than in the following periods due to extensive project planning activities (Black et
al., 2009, p. 42).
Sarah Otyepka, Benjamin Mosig, Marco C. Meier
Period t1 Period t2 Period t3 Periods t4 ... tn
Agile
method A
Benefits € --- € 5,000 € 7,500 € 9,000
Costs € 100,000 € 40,000 € 60,000 € ---
Plan-
driven
method P
Benefits € --- € --- € --- € 9,000
Costs € 80,000 € 60,000 € 60,000 € ---
Legend Design Implementation Testing Productive System
Figure 2: Comparison of benefit and cost cash flows for A and P along the project
roadmap
The risk situation is characterized as follows. In case of a changing environment, we as-
sume additional costs of €30,000 for A and €60,000 for P since the latter one is not able
to adjust to changes as flexible. The likelihood for the occurrence is identical for both
project management methods. In case of improper system integration, we assume addi-
tional costs of €180,000 whereby the likelihood of this peril is three times higher for A
than for P.
The project roadmap has the milestones as depicted in Figure 2. Note that we only take
into account three periods in the proposed model. We additionally show the final BI sys-
tem (periods t4 to tn) for the sake of completeness. In case of applying A, in the first itera-
tion the most important steps are taken, namely developing the major parts of the DW,
integrating ODB1 and ODB2, and establishing R1. In the second iteration, the DW is
completed and R2 is established. The missing ODB3 and R3 are integrated in the last
iteration. In case of applying P, the first period is used to design the system architecture,
the second period is used for the initial implementation of all system components, and in
the last period the system is tested.
Formally, the decision model is based on following assumptions:
The decision to implement a BI system is final. The company only chooses be-
tween two project management methods, agile (A) or plan-driven (P). Both are
capable of delivering the same functionality within the same time (Black et al.,
2009, p. 42). Due to its distinct strengths and weaknesses, each project manage-
WSBI 2012 – Agile and Plan-driven Project Management in a Business Intelligence Context 43
ment method has a different cost/benefit structure that is captured in aggregated
cash flows per period.
All considered cash flows are discounted to the period the project starts.
The project is affected by two types of risks only. These are the likelihood of en-
vironmental changes within a project period δ ϵ [0; 1] and the likelihood of a lack
of integration at the end of the project ε ϵ [0; 1]. Both risks are independent.
δ is constant for all periods and independent of the chosen project management
method. ε is inherent to the chosen project management method and hence dif-
fers. δ occurs in each period, ε only occurs in the last one.
If a risk occurs, the company will fix the failure. The costs associated with the
correction are considered within the cash flow of the period the risk occurred
(again discounted to the period of project start).
The assumptions required for applying a NPV approach are fulfilled. That is, the
company reinvests all cash flows in other value-contributing projects and acts
within a perfect capital market.
Due to the risk structure of the project, we model the associated risks by a random exper-
iment with three periods. Figure 3 shows an exemplary decision tree for A. In each of the
three periods, two outcomes regarding the environmental risk are possible, “no change”
with a likelihood of 1 - δ, and “change” with a likelihood of δ. Additionally, in the last
period the risk of improper integration occurs with a likelihood of ε while ”good“ integra-
tion occurs with a likelihood of 1 - ε. According to A5, we consider both risks by adding
their expected costs to the cash flow of the period the risk occurs.
Period t1 Period t2 Period t3
"Good"
1-ε integration
No changes (€ -43,388)
1-δ
(€ -43,388) Improper
ε integration
1-δ No changes (€ -192,149)
(€ -31,818)
No changes "Good"
1-δ (€ -100,000) 1-ε integration
δ Changes (€ -68,128)
Project Changes (€ -68,128) Improper
start δ (€ -59,091) ...
ε integration
Changes (€ -216,942)
δ ...
(€ -130,000)
Figure 3: Decision tree showing the discounted cash flows for A
Sarah Otyepka, Benjamin Mosig, Marco C. Meier
To calculate the NPV, the likelihoods have to be multiplied along all possible paths in the
decision tree. The expected cash flow is this likelihood of a path multiplied by the sum of
the corresponding cash flows along the path (already discounted based on an internal
interest rate of 10%). The total NPV is the sum of these expected cash flows of all possi-
ble paths.
Likelihood of environmental changes
δ 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
ε
Legend
0
Likelihood of a lack of integration
0.1 0-5 %
0.2 5-10 %
0.3
10-15 %
0.4 Agile project
management method A 15-20 %
0.5
0.6 20-25 %
Plan-driven project
0.7 management method P 25-30 %
0.8
30-35 %
0.9
1 >35 %
Figure 4: Result of the decision model applied to the demonstration example
According to the common NPV approach, one compares the NPVs of two alternatives
and chooses the higher one – given it is positive (Ross, 1995, p. 96). In our demonstration
example, the resulting NPV is expected to be negative since we only take into account the
cash flows during project runtime but do not consider returns that are generated after the
end of the project. This is possible since A1 ensures that the project is economically fea-
sible in the long term. Furthermore, we extend this NPV approach by a comparison of the
NPVs of both project management methods. We simulate all combinations of δ and ε in
discrete steps of 0.1 in order to identify the risk parameter combinations that mark the
turning point in the decision. Figure 4 visualizes the resulting recommendations for any
parameter combination.
We see that the model’s recommendation is not contrary to an intuitive estimation. For
highly agile environments with δ 0.8, A is superior regardless of the value of ε. A is
also superior in case of low integration risks with ε 0.3. For ε 0.6 and δ 0.5, P is
the most appropriate project management method. For all other parameter combinations,
the turning point is located in an area of indifference between both project management
methods. The shading of Figure 4 indicates the deviation of the resulting NPV in percent
if applying the less appropriate project management method instead of the recommended
one. While the decision’s turning point is located in a corridor of marginal differences, it
reaches up to 40% in the boundary area. This result shows that applying project manage-
WSBI 2012 – Agile and Plan-driven Project Management in a Business Intelligence Context 45
ment methods inconsiderately might result in significant discrepancies in the resulting
NPVs. Even though the model does not yet offer surprising findings concerning a rec-
ommendation, we appreciate that its results conform to intuitive decisions in this simple
demonstration example. This indicates that a more advanced model might give useful
recommendations in more complex decision situations when an intuitive decision is no
longer reliable.
The proposed decision model requires a sufficient clarity and preciseness of parameter
values that might demand great efforts to be provided. To analyze the necessary degree of
data accuracy, future research will include sensitivity analyses addressing the parameters
of the model. These sensitivity analyses should further strengthen the fact base on which
decision makers can derive a recommendation for the appropriate project management
method – instead of relying on “one-size-fits-all” instructions or gut feeling. In this re-
spect, this paper already provides a rough model sketch and proves the principle feasibil-
ity of an approach based on estimating two risk parameters and a project’s cash flows.
4 Limitations and Outlook
The presented model fulfills or is able to fulfill all six initially stated requirements men-
tioned in section 2. (1) It is applicable to a class of problems, namely to decisions on the
project management method in a BI project. (2) It contributes to the body of knowledge
since no formal quantitative model on the decision between agile and plan-driven project
management methods could be found in literature. (3) The underlying NPV approach is
established and results are easily reproducible. (4) The model claims to add value to the
decision process by supporting decision makers in the selection of the most appropriate
project management method. (5) Since estimating the project’s cash flows is necessary
for project planning anyway, the model is easily applicable. Decision makers have to only
estimate two additional risk parameters. (6) Relying on a NPV-based decision model, we
propose the way towards a quantitative, value-based and future-oriented approach.
Nevertheless, the proposed model is beset with limitations that need to be taken into ac-
count when applying it in project settings.
The proposed model is only applied to a three period case study.
The proposed model only takes into account two decisive parameters.
The peril of improper integration is assumed to occur only in the last period,
while other arguments suggest this risk could be prevented earlier.
Sarah Otyepka, Benjamin Mosig, Marco C. Meier
The proposed model only takes into account a “pure-blood” agile and plan-driven
approach and neglects other project management methods and hybrids of both ex-
tremes.
The proposed model claims to offer a clear recommendation for a decision, yet
might underlie errors regarding the estimation of cash flows and risk parameters.
Despite the named limitations, the proposed NPV model represents an initial decision
model comparing the impact of selected project management methods in a BI context.
Based on the presented model sketch, we currently work on a more detailed approach.
We will address L1 by the development of a general analytical model that can be applied
to any number of project periods. The limitation to two parameters (L2) might be consid-
ered a far reaching simplification. Therefore, based on a more detailed analysis it might
be an interesting option to add more risk parameters as, for example, the five parameters
identified by Boehm and Turner (2003). Referring to L3, we will consider the possibility
that decision makers detect an integration problem not at the end, but earlier during the
project. Thus, an extended model might consider both risks in each period. Additionally,
further research should address L4 by proposing a general analytical model that can be
applied to any project management method. To strengthen the reliability of the decision
model and to examine the effects of estimation errors, we plan to address L5 by conduct-
ing sensitivity analyses to test the impact of deviations of estimated cash flows and risk
parameters. Additionally, it might be interesting to evaluate the model using case studies
from different industries in order to identify if – and in which aspects – the model needs
to be adapted. Summarizing, since there seems to be a lack of theoretical foundation for a
decision on the appropriate project management method in a BI project, we presented a
research-in-progress model already giving first recommendations and an outlook on fu-
ture research.
References
Abrahamsson, P., Conboy, K., & Wang, X. (2009). ‘Lots done, more to do’: the current
state of agile systems development research. European Journal of Information
Systems, 18(4), 281-284.
Adamala, S., & Cidrin, L. (2011). Key Success Factors in Business Intelligence. Journal
of Intelligence Studies in Business, 1(1), 107-127.
Black, S., Boca, P. P., Bowen, J. P., Gorman, J., & Hinchey, M. (2009). Formal versus
Agile: Survival of the Fittest. IEEE Computer, 42(9), 37-45.
WSBI 2012 – Agile and Plan-driven Project Management in a Business Intelligence Context 47
Boehm, B., & Turner, R. (2003). Balancing Agility and Discipline: A Guide for the
Perplexed Addison-Wesley Longman Publishing Co., Inc.
Charvat, J. (2003). Project Management Methodologies: Selecting, Implementing, and
Supporting Methodologies and Processes for Projects John Wiley & Sons Inc.
Cockburn, A. (2008). Using Both Incremental and Iterative Development. STSC
CrossTalk, 21(5)
Coenenberg, A. G., Mattner, G. R., & Schultze, W. (2003). Wertorientierte Steuerung:
Anforderungen, Konzepte, Anwendungsprobleme. In A. Rathgeber, H. J. Tebroke &
M. Wallmeier (Eds.), Finanzwirtschaft, Kapitalmarkt und Banken, Festschrift für
Professor Dr. Manfred Steiner (1st ed., pp. 1-24). Stuttgart: Schäffer-Poeschel.
Conboy, K. (2009). Agility from first principles: reconstructing the concept of agility in
information systems development. Information Systems Research, 20(3), 329-354.
Fowler, M., & Highsmith, J. (2001). The Agile Manifesto. Software Development, 9(8),
28-35.
Höhn, R., & Höppner, S. (2008). Das V-Modell XT: Grundlagen, Methodik und
Anwendungen (1st ed.) Springer Publishing Company, Incorporated.
Kerzner, H. (2009). Project Management: a Systems Approach to Planning, Scheduling,
and Controlling (10th ed.) John Wiley & Sons.
Larman, C., & Basili, V. R. (2003). Iterative and Incremental Development: A Brief
History. IEEE Computer, 36(6), 47-56.
Maruping, L. M., Venkatesh, V., & Agarwal, R. (2009). A Control Theory Perspective on
Agile Methodology Use and Changing User Requirements. Information Systems
Research, 20(3), 377-399.
Mertens, P. (1999). Operiert die Wirtschaftsinformatik mit den falschen
Unternehmenszielen?-15 Thesen. In J. Becker, W. König, R. Schütte, O. Wendt & S.
Zelewski (Eds.), Wirtschaftsinformatik und Wissenschaftstheorie: Bestandsaufnahme
und Perspektiven (pp. 379-392) Gabler Verlag.
Moss, L. (2009). Beware of Scrum Fanatics On DW/BI Projects. EIMI, 3(3)
Moss, L., & Atre, S. (2003). Business Intelligence Roadmap: The Complete Project
Lifecycle for Decision-Support Applications Addison-Wesley Professional.
Österle, H., Becker, J., Frank, U., Hess, T., Karagiannis, D., Krcmar, H., et al. (2010).
Memorandum zur gestaltungsorientierten Wirtschaftsinformatik. Schmalenbachs
Zeitschrift Für Betriebswirtschaftliche Forschung, 62(9), 662-672.
Sarah Otyepka, Benjamin Mosig, Marco C. Meier
Project Management Institute. (2008). A Guide to the Project Management Body of
Knowledge (4th ed.) Project Management Institute.
Project Management Methodologies. (n.d.). Retrieved July 15, 2012, from
http://infolific.com/technology/methodologies/
Ross, S. A. (1995). Uses, Abuses, and Alternatives to the Net-Present-Value Rule.
Financial Management, 24(3), 96-102.
Wysocki, R. K. (2011). Effective Project Management: Traditional, Agile, Extreme (6th
ed.) John Wiley & Sons.