=Paper=
{{Paper
|id=None
|storemode=property
|title=Agile in Public Administration: Oxymoron or Reality? An Experience Report.
|pdfUrl=https://ceur-ws.org/Vol-1017/Paper1CAiSE_IT2013.pdf
|volume=Vol-1017
|dblpUrl=https://dblp.org/rec/conf/caise/SalinasSEM13
}}
==Agile in Public Administration: Oxymoron or Reality? An Experience Report.==
https://ceur-ws.org/Vol-1017/Paper1CAiSE_IT2013.pdf
Agile in Public Administration: Oxymoron or reality? An
experience report
C.J. Torrecilla-Salinas1, J. Sedeño1, 2, M.J.Escalona1 and M. Mejías1
1
Department of Computer Languages and Systems. University of Seville
{carlos.torrecilla,jorge.sedeno}@iwt2.org
{mjescalona,risoto}@us.es
2
Agencia Andaluza de Instituciones Culturales
jorge.sedeno@juntadeandalucia.es
Abstract. In the last 10 years, Agile methods and practices have emerged as an
alternative for software development. Different "flavors" of Agile have
appeared ranging from project management to tests organization. These
approaches have being gaining popularity and involve now a solid option for
organizations developing software, but what about Public Administrations? Is
Agile a suitable option for developing software in Public Administrations?
Even if Public Administrations have been traditionally regarded as change-
resistant, Agile approach can also provide them with the benefits of quick
adaptation and frequent value delivery.
This paper presents the results of two different projects, which use an Agile
framework based on Scrum, developed by a Spanish Public Administration.
Additionally, after considering the obtained results, it takes out some relevant
learned lessons on the suitability of applying Agile approaches to Public
Administration environments.
Keywords: Agile methodologies, Scrum, e-Government, Public Administration
1 Introduction
The appearance of methodologies like Scrum [1] or eXtreme Programming [2] during
the nineties, with their iterative and incremental approach to software development,
represented an enormous change in the way systems were developed. Some years
before their appearance, a group of the most recognized practitioners of these
methodologies published what was known as “Agile manifesto” [3], pointing to the
main values of Agile: working software over documentation, collaboration over
contract negotiation, response to change over following plans and people and
individuals over tools. Based on these guidelines, the basic principles of Agile
include, among others: early delivery of value; quick response to change, even on late
phases of development; close collaboration between the development team and the
business representatives or periodic revision of the development process.
Agile can be conceived as a label where several methodologies and techniques,
sharing its principles, can be included. Some of them are Scrum [1], eXtreme
Programming [2], Crystal [4], Lean Software Development [5] or Feature Driven
Development (FDD) [6]. From the set of Agile techniques, the most popular is Scrum
[7], an incremental and iterative framework for product development proposed by
Ken Schwabber and Jeff Sutherland [8] and inspired by the work of Takeuchi and
Nonaka [9]. Using Agile methodologies is being popular in the last years [10] [11],
becoming a solid alternative for systems development, especially for those which
demand early delivery of value and quick-response to changes. Nevertheless, despite
Public Administrations have been traditionally conceived as change-resistant
organizations, the challenges they have to face up in the actual interconnected and
global economy make them need faster adaptation to the changing citizens’
requirements.
Based on the foregoing, this work presents the results of applying an Agile
framework, based on Scrum, to two different projects developed by a Spanish Public
Administration, with the following objectives:
• Evaluating the possibilities of applying Agile methodologies to a high
regulated environment.
• Taking out, comparing the results of both projects, the relevant lessons
learned and the future lines of research.
This paper is organized into the following sections: After this introduction, Section 2
describes the environment where the projects were developed and Section 3
overviews the proposed Agile framework. Then, section 4 analyzes and compares the
results obtained from both projects and, finally, Section 5 states the main conclusions
and lessons learned.
2 The environment
Projects presented in this work were developed by the Ministry of Culture and Sports
of the regional government of Andalusia, the largest autonomous region in Spain. The
autonomous government of Andalusia is called Junta de Andalucía and it is organized
into eleven ministries, being the Ministry of Culture and Sports one of them. It
develops policies in the cultural area, such as the management of public libraries,
museums and archives, and supports the practice of sport in the region. In addition, it
promotes and supports regional cultural and sports industries. The ICT (Information
and Communication Technologies) department, within the Ministry, is responsible for
developing the projects described in the present paper. It encourages all technological
projects and policies within the Ministry, including both the development of new
software systems and the maintenance of ICT infrastructure.
The ICT department, in liaison with the IWT2 research group, created a Project
Management Office (PMO) [12] in order to improve the project management
capabilities, optimize the way projects are managed and reduce costs and effort by
implementing a scale economy. Some of PMO’s tasks entail improving the way
projects were managed as well as the research on new project management
methodologies. Thus, it was proposed to start with pilot projects using an Agile
framework based on Scrum, to test its suitability and feasibility, so that it can be
generally applied in the Ministry. The two selected projects were:
• eBOJA: It deals with adapting and integrating the Ministry systems to using
the new electronic official journal of the regional government of Andalusia.
It included, among others points, the design of the administrative procedure,
the deployment of the two Web applications within the Ministry’s
infrastructure, the development of some software APIs to protect the internal
systems of the Ministry, the interconnection of the applications with the
general infrastructure and all the aspects related to change and users’
expectations management.
• TOPOS: This was an infrastructure project which gathered the following
objectives: Reorganizing the environments where the Ministry’s systems
were deployed; standardizing the software products used to support the
systems (operative systems, databases or application servers, for example);
updating the versions of some homemade software products; cleaning and
decommissioning some obsolete systems and getting high-availability
configuration of all e-Government services.
eBOJA was a development and integration project carried out by a team of four
members, under a strict and short deadline fixed by a law approved by the regional
government. It involves several internal and external stakeholders. Despite the
development team had long experience working together, it was the first time they
worked with some of the technology concerned. TOPOS, in contrast, was an
infrastructure project, with a flexible deadline and requirements, and an indirect
business impact. The team in charge of carrying it out comprised five members, with
less team dynamics than eBOJA’s team, and, in this case, they were very familiar
with the technology.
Both projects were developed with internal staff, being the first experience with Agile
techniques for almost all participants. These projects were developed in 2012; eBOJA
lasted almost 4 month and TOPOS almost a year.
3 The Agile framework
An Agile framework composed of several techniques was used to develop and
manage both projects. The main techniques are listed below:
• Scrum: The Scrum lifecycle was the basis of the project management
methodology. The development process was divided into time-boxed
iterations (lasting from 3 to 4 weeks, depending on the project). A Product
Backlog, including all the features of the project, was created for each
project in order to guide such process. Each Sprint included a Sprint
planning meeting, a Sprint review, a Sprint retrospective, using Agile
retrospective techniques [13] and a “Product Backlog grooming” [14]
meeting. As in both cases the teams were located in different places, they
were not able to organize Daily Scrum meetings. Consequently, tools like IM
message, chats and ticketing systems were used to cope with this point.
• Agile estimating and planning techniques: User stories [2] [15], story
points [16] and value points [17] were used to estimate and plan the features
of each project. Each Sprint the project plan was reviewed, creating a
dynamic project plan, which was able to include new users’ needs that could
appear during the development process.
• Agile EVM: An Agile approach to Earned Value Management techniques
was used [18] to control the costs, effort and planned schedule of the
projects, without losing the ability to change. This approach is based on the
use of two indexes, the Cost Performance Index (CPI) and the Schedule
Performance Index (CSPI). The CPI is the result of dividing the Earned
Value of the project, calculated as the monetary cost of the finished story
points in a certain Sprint, by the Actual Cost, calculated as the monetary cost
of the real working hours in a certain Sprint. The CSPI is the result of
dividing the Earned Value of the project, calculated as the monetary cost of
the finished story points in a certain Sprint, by the Planned Value, calculated
as the monetary cost of the planned story points for a certain Sprint.
Decisions were made attending to these values with the aim of improving the
way the project was conducted.
• Agile productivity metrics: A set of productivity metrics, proposed by
Downey and Sutherland [19], were used to help the team improve through
the Sprints. These metrics allowed knowing, among other things, how much
of the team’s work is really delivering a value to the customer, how much the
team is improving through the Sprints of a project and how good or bad are
the estimations.
Figure 1 summarizes the proposed Agile framework:
Fig. 1. Proposed Agile framework.
4 Results of the projects
In this section, we will list the results of both projects in order to be able to compare
them. Using Agile estimating and planning techniques, an initial plan, including,
according to McConnell’s uncertainty cone [20], an uncertainty of 25% was
developed and approved. Table 1 shows the initial plan of each project against the real
results. Regarding the presented data, it has to be mentioned that, as both projects
were developed using internal resources, no real expenditures were performed. All
economic data offered are only assumed estimations to better manage the projects.
Table 1. Results against estimates in both projects
TOPOS Project eBOJA project
Magnitude Initial forecast Final value Initial forecast Final value
Total story points 70±18 64 56±14 56
Velocity 7±2 5,8 11±3 11.20
Number of iterations 10±3 11 5±1 5
Sprint length (week) 4 4 3 between 3 and 4
Project length 40±12 weeks 53 weeks 15±3 weeks 16 weeks
Hours per Sprint 137±34 67,72 150±38 111.8
Hours per Project 1,370±340 744,94 764±191 559
Average cost per Hour € 25,90 N/A € 24,62 N/A
Cost per iteration € 3,584.30 ± 887.01 € 1,777.74 € 3,694.24±€ 923.59 € 2,752.52
Total project cost € 35,843.00 ± 8,870.10 € 19,293.95 € 18,807.55±€ 4,701.89 € 13,762.58
It can be observed that in both cases, most of the estimations are in-line with the real
data (except for the case of the expected hours and cost of TOPOS Project), being
eBOJA’s estimations more precise. It can be explained by the fact that the team
working in eBOJA had some previous experience working together as a team. Also
the fact that eBOJA lasted less than TOPOS, can explain the better estimations. In
both cases, effort and cost estimations were higher than the real values, probably due
to the teams’ lack of experience and a self-protection tendency to get enough time to
finish the tasks. It has to be mentioned that these initial estimations were reviewed
and corrected at the end of each Sprint, according to the result. Moreover, the CPI
index measures the difference between estimated and real costs, in our case, at the
Sprint level. Figure 2 shows the evolution of CPI for TOPOS project and Figure 3
represents the evolution of CPI for eBOJA project:
Fig. 2. TOPOS’ CPI evolution. Fig. 3. eBOJA’ CPI evolution.
As it can be observed, both CPI indexes started below 1, meaning that the cost was
higher than the earned value. However, they were increasing through the Sprints in
both projects. In the case of TOPOS project, it increased and stabilized around 1,
probably due to the learning process of the team during the project. In contrast, in
eBOJA, the estimations were mainly above 1, probably because of a certain tendency
to self-protection. The fact that TOPOS be a longer project also helped this team to
gradually learn, but it did not happen in eBOJA because of its duration. Moreover, the
CSPI index measures the relation between the estimated schedule and the real one, in
our case, at a Sprint level. Figure 4 shows the evolution of CSPI for TOPOS project
and Figure 5 represents the evolution of CSPI for eBOJA project:
Fig. 4. TOPOS’ CSPI evolution. Fig. 5. eBOJA’ CSPI evolution.
In both cases, CSPI started below 1 and evolved to be around 1 in the end of the
project. The learning process can be observed in both cases, resulting it quicker in the
eBOJA project, probably due to the previous experience already mentioned and some
legal constraints to deliver the project. The slow evolution of TOPOS team could be
also caused by external factors (as this was an infrastructure project, not a business
project and sometimes it was affected by the priorities of other projects) and by its
longer duration. Lastly, Table 2 shows the average results of the selected productivity
metrics for each of the projects:
Table 2. Global comparative productivity metrics between TOPOS and EBOJA
Metric TOPOS EBOJA
Sprints 11 5
Average Velocity 5,8 11,2
Average Hours per Story Points 11,73 9,96
Average Velocity in Hours 87,65 126,27
Average Work Capacity 65,51 111,5
Average Focus Factor 133,80% 113,25%
Average % of Accepted Work 56,59% 85,95%
Average TVI+ 118,62% 119,82%
Table 2 shows that eBOJA team spent more hours per Sprint (its average Velocity in
Hours and its average Work Capacity are higher), and also its percentage of accepted
work is higher (a higher number of the dedicated hours was used to deliver real value
to the user). This means that eBOJA team was more productive than TOPOS team,
they had a higher work capacity and they delivered more value. This responds to the
fact that they had more experience working together and also to the urgency of their
project. Both teams were able to improve through Sprints, as it can be observed in
their average TVI+, which is very similar for both projects. Therefore, both teams
were able to better deliver at the end of the project than at the beginning. It seems to
have sense, for at the end of the project they better knew both, the methodology and
their colleagues. Focusing on estimations, both teams tended to overestimate, as they
show a focus factor over 100% in both cases. As it was said, this fact can be caused
by a tendency to self-protection.
5 Lessons learned and future work
This paper has presented the results of applying an Agile framework for project
management in two projects developed by a Public Administration. We have realized
that the approach allowed the teams to develop an initial project plan that can drive
the project, and also to correct it in terms of the results, using the values CPI and
CSPI indexes as guidance. Besides, the approach provides a relevant view on the
status of the project, which becomes very useful, both for the team and for its
management. It can also be regarded as a learning process, in which the team is
improving its estimation skills. Additionally, the framework transforms the estimation
and planning effort into continuous and participative processes involving the whole
team, instead of being an initial phase of the project. The techniques used seem to be
also very useful too, particularly to compare the performance of different teams,
which will be very beneficial for organizations dealing with several projects. Lastly,
the methodology seems to be transversal and may be applied to different kind of
projects (development, integration or infrastructure, for example). Nevertheless, in the
case of the analyzed projects, the fact that the team was not co-located is considered
an impediment to improve communication and collaboration. Furthermore, both
teams tended to overestimate their work, maybe with self-protection intentions and
because this was their first Agile experience. Besides, as a conclusion, the approach
works better in shorter projects and with more experienced teams.
A main conclusion is that Agile can be a suitable approach for certain projects in
Public Administration. In the context of the research group, we worked with the
Ministry of Culture and Sport during the last years in Web Engineering projects.
However, previous experiences were not developed with Agile solutions. This
experience opens a new research line to look for an approach to develop a Web
Engineering approach in Agile context. Generalizing and formalizing this approach
could be useful to face up short projects with changing requirements and needs of
early deliver of value.
Acknowledgements.
This research has been supported by the Tempros project (TIN2010-20057-C03-02),
and by the project NDTQ-Framework (TIC-5789) of the Junta de Andalucía, Spain.
We would also like to thank the Ministry of Culture and Sport of Junta de Andalucía
for letting us issuing these data.
References.
1. Schwaber, K. “Scrum Development Process”. Oct. 1995. In proceedings of the 10th
Annual ACM Conference on Object Oriented Programming Systems, Languages and
Applications (Austin, TX, USA 15-19 Oct. 1995). OOPSLA ‘95. ACM.
2. Beck, K. “Extreme Programming Explained: Embrace Change”. Boston: Addison-Wesley.
1999.
3. Beck, K. et al. Manifesto for Agile Software Development. 2001. From
http://www.agilemanifesto.org. Last accessed 01-04-2013.
4. Cockburn, A. “Crystal Clear: A Human-Powered Methodology for Small Teams”. Boston:
Addison-Wesley. 2004.
5. Poppendieck, M.; Poppendieck, T. “Lean Software Development. An Agile Toolkit”.
Boston: Addison-Wesley. 2003.
6. Palmer, S. R. “A Practical Guide to Feature-Driven Development”. NJ: Prentice-Hall.
2002.
7. Pikkarainen, M. et al. “The Impact of Agile Practices on Communication in Software
Development”. Empirical Software Engineering, Springer, pp 303-337. May. 2008.
8. Sutherland, J.; Schwaber, K. “The Scrum Guide: The Definitive Guide to Scrum: The
Rules of the Game”. 2011. From http://www.scrum.org/Scrum-Guides. Last accessed 01-
04-2013.
9. Takeuchi, H.; Nonaka I. “The New Product Development Game”. Harvard Business
Review 64(1): 137-146. Jan.-Feb. 1986.
10. Ambler, S.W. “Lessons in Agility from Internet-based Development.” IEEE Software, pp.
66-73. Mar-Apr, 2002
11. Begel, A.; Nagappan, N. “Usage and Perceptions of Agile Software Development in an
Industrial Context: An Exploratory Study. In proceedings of the 1st International
Symposium on Empirical Software Engineering and Measurement (Madrid, Spain,
September 21-27 2007) ESEM '07, IEEE.
12. IWT2. 2013. From http://www.iwt2.org. Last accessed 01-04-2013.
13. Derby, E.; Larsen, D. "Agile Retrospectives. Making Good Teams Great". Dallas: The
Pragmatic Bookshelf. 2006.
14. Cohn, M. “Succeeding with Agile Using Scrum”. Boston: Addison-Wesley. 2009.
15. Cohn, M. “User Stories Applied: For Agile Software Development”. Boston: Addison-
Wesley. 2004.
16. Cohn, M. “Agile Estimating and Planning”. NJ: Addison-Wesley. 2005.
17. Highsmith, J. “Agile Project Management: Creating Innovative Products, Second Edition”.
NJ: Addison-Wesley. 2009.
18. Sulaiman, T. “AgileEVM: Measuring Cost Efficiency Across the Product Lifecycle”.
2007. From http://www.infoq.com/articles/agile-evm. Last accessed 01-04-2013.
19. Downey, S.; Sutherland J. “Scrum Metrics for Hyperproductive Teams: How They Fly like
Fighter Aircraft”. In proceedings of the 45th Hawaii International Conference on System
Science. 2012. (Maui, Hawaii, USA, January 4-7 2012) HICSS 2012.
20. McConnell, S. Software Projects Survival Guide. Redmond: Microsoft Press. 2009.