<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Upping the Ante: Agile Product Development at RAY</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Janne J. Korhonen</string-name>
          <email>janne.korhonen@aalto.fi</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Konsta Luhtala</string-name>
          <email>konsta.luhtala@ray.fi</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Aalto University School of Science</institution>
          ,
          <addr-line>Espoo</addr-line>
          ,
          <country country="FI">Finland</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Finland's Slot Machine Association (RAY)</institution>
          ,
          <addr-line>Espoo</addr-line>
          ,
          <country country="FI">Finland</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Finland's Slot Machine Association (RAY) has recently undergone a major transformation from a steady operator of slot machines to an agile innovator of new games and concepts. As a key part of this transformation, agile methods used in product development teams have been scaled up to the organizational level using Scaled Agile Framework (SAFe). To help analyze the development in the organization's product development capability, we outline the Agile Product Development Maturity Framework (APDMF) that addresses the type of product market, complexity of work, the nature of development process, and the scope of agile approach. We argue that SAFe provides RAY with a “scaffolding” to support this transformation and to institutionalize agile product development for the future.</p>
      </abstract>
      <kwd-group>
        <kwd>agile</kwd>
        <kwd>product development</kwd>
        <kwd>levels of work</kwd>
        <kwd>enterprise transformation</kwd>
        <kwd>case description</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        The general objective of this study is to contribute to the understanding of
prerequisites of implementing agile product development. Using a single case approach, we
analyze how agile product development was implemented at Finland’s Slot Machine
Association (Raha-automaattiyhdistys, RAY) to come to grips with the progressively
complex product market of the organization. To structure our analysis, we outline
Agile Product Development Maturity Framework (APDMF) that helps explain how
the product development capability of the organization has recently transitioned to an
essentially new level of complexity and coherence. This “level of work” [e.g. 1,2] is
commensurate with RAY’s evolving product market [cf. 3] that entails technological
change and evolving preferences of customers. In line with the observation that
formalized new service development processes are an antecedent of new service
development competence [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], SAFe (Scaled Agile Framework) [
        <xref ref-type="bibr" rid="ref5 ref6">5,6</xref>
        ] provides a strategic
system that helps integrate product development efforts across organizational teams
and units. Coordination of cross-functional expertise through SAFe, augmented by
adequate information and communication technology capability, is expected to yield
higher product development performance [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ].
      </p>
      <p>The structure of the paper is as follows: In Section 2, we review the business
context of RAY that prompted the transformation in its product development capability
and the introduction of the SAFe framework. To provide a theoretical framework to
assess this transformation, we review three strands of literature in Sections 3 through
5. In Section 3, we introduce the notion of “Levels of Work” – a normative
stratification of complexity that underlies human work. Section 4 presents a typology of three
product markets of increasing dynamism and respective approaches to product design.
And in Section 5, we discuss the three levels of “agile enterprise big picture” that
underlies the SAFe framework. An integration of the theoretical background
presented in Sections 3–5 is provided in Section 6 that puts forward the Agile Product
Development Maturity Framework (APDMF), a framework that is intended to help
assess an organization’s product development maturity and inform its further
development. In Section 7, we review the implementation of agile product development at
RAY, analyzing it, ex post, against the backdrop of this framework. Finally, we
conclude the paper with discussion and reflection in Section 8.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Business Context</title>
      <p>RAY offers entertaining games in about 20,000 physical slot machines around
Finland, in restaurants, arcades, online, and at Casino Helsinki. It is a non-profit special
organization, governed by legislation and decrees, which give it the exclusive right to
operate slot machines, Internet casino games, and physical location casino activities in
Finland. The profits from RAY’s games are channeled to a wide range of
organizations promoting health and social welfare. The fund allocation is guided by policies
created by RAY’s Board of Directors and as agreed with the Ministry of Social
Affairs and Health. RAY is a responsible operator that also ensures players’ legal
protection, prevents misuse and crime, and reduces the harmful social effects of gaming.</p>
      <p>In the wake of 2000’s, a number of trends suggested that the organization should
transform the way in which it conducts business. Firstly, electronic means of payment
started to challenge traditional cash payment. Secondly, international online casinos
emerged and very quickly found their audience irrespective of national borders. At
RAY, the need was felt to create a responsible, local alternative that is safe and
reliable compared to many overseas online casinos. Moreover, Veikkaus, another Finnish
organization governed by the Lotteries Act, had already embraced the digital channel.
Thirdly, the new generation of potential players started to have growing expectations
for the functionality and user experience of slot machine games. The good old,
triedand-tested games needed to be updated to the new millennium.</p>
      <p>In the early years of the 2000’s, RAY experimented alternative means of payment
and around 2005 decided to furnish its slot machines with debit card terminals. This
decision was the turning point towards RAY’s capability of producing contemporary
digital consumer services. The deployment of new devices started in 2009. Rolling
out the entire installed base of about 20,000 machines was a major undertaking that
took five years. In addition to the debit card terminals, many other features were
implemented during 2010–2012: new types of games as well as updates and
configuration over network. The required new payment transaction system was a challenge in
its own right, but the new slot machines also needed to be always online.
Authentication of the player was enabled at slot machines as well as in online gaming services.
So far the feature has made it possible to set individual limits for playing and to
benefit from exclusive perks. Today, there are over 300,000 registered customers. Parallel
to the development in slot machine games, a new decree allowed RAY to establish an
Internet casino in 2009/2010. The Web also provided a new channel for games.</p>
      <p>By the early 2010’s, the product development teams of RAY had already attained a
reasonable level of maturity in agile software development methods, but steering of
product development and business units was not commensurate with agility in teams
that worked separately and in different rhythms. As the number of product
development projects increased from 2011 onwards, an increasingly large part of projects
called for creating crosscutting capabilities related to RAY’s gaming systems. The
amount and complexity of projects brought about the need to improve design and
steering of product development. The management of product development realized
that without agile practices extended to budgeting, concepting, and decision-making,
agile product development and deployment would not develop enough to account for
the future needs. As a framework that would support large-scale agile product
development, RAY decided to adopt SAFe, a model that helps roll out enterprise-wide
agile methods for software-based product development.
3</p>
    </sec>
    <sec id="sec-3">
      <title>Levels of Work Complexity</title>
      <p>
        Each organization has its unique structure, with an idiosyncratic number of
organizational levels. However, according to late organizational psychologist Elliott Jaques [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]
there is an underlying deep structure that determines the “requisite” number of levels,
contingent on the complexity of the environment, which may or may not reflect the
actual stratification of an organization. Jaques recognized that this hierarchical
ordering of work complexity, termed Requisite Organization (RO), reflects the
discontinuous developmental stages in the nature of human capability. The role complexity
increases in discontinuous steps, stratifying varying kinds of work into natural layers, or
“requisite strata”, in the organization. The following labels epitomize these Levels of
Work [
        <xref ref-type="bibr" rid="ref8 ref9">8,9</xref>
        ]:
      </p>
      <p>I. Quality: excellence of task.</p>
      <p>II. Service: effective coordination, continuous improvement, efficiency.
III. Practice: work practices and systems, productivity.</p>
      <p>IV. Strategic Development: innovation, change and continuity.</p>
      <p>V. Strategic Intent: direction, profit, long-term viability.</p>
      <p>VI. Corporate Citizenship: vision, building strong national and world wide
presence.</p>
      <p>VII. Corporate Prescience: new forms of social, political and economic
institutions.</p>
      <p>In the following, we will briefly review three of these levels, Strata III–V, which
represent middle and top management levels in a typical self-governing organization
(of Str-V complexity) such as a large single-organization business or an independent
strategic business unit of a large corporation organization. We will also discuss the
dynamics of transition between these levels, as the organization grows in complexity.
3.1</p>
      <sec id="sec-3-1">
        <title>Stratum III</title>
        <p>
          At Stratum III, work is about systematic provision [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] to the varying needs of today
and in extrapolative anticipation of those of tomorrow. The focus is on designing and
optimizing individual work systems that cope with known or predictable situations
[
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. The response at this level is to develop systems to maximize the efficiency of
resources to handle a fluctuating workload [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] and to encompass genuine
openended cases [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ]. Such systems are likely to involve the design or redesign of work
processes from a number of work streams [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ].
        </p>
        <p>
          Stratum III is about creating value in the present within the existing asset base;
there is no expectation for investing new capital for innovation in new products, new
services, and new businesses, but the decision-making authority is limited to
shortterm core business process efficiencies to maximize return on investment [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ].
Technical improvement and innovation may be of breakthrough nature, but change does
not represent discontinuation to the current practice but rather new ways of organizing
and utilizing given resources [
          <xref ref-type="bibr" rid="ref2 ref8">2,8</xref>
          ].
        </p>
        <p>
          The unit managers at Stratum III typically manage a mutual recognition unit, such
as a department of the organization, up to 300 people [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]. Managers at this level make
the most of technological, people and financial opportunities to best meet local
conditions [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. Non-managerial roles at this level of work complexity include “senior” or
“chief” engineers, scientists, and many lawyers and doctors [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]. Examples of work
include: setting up a training program; developing a new treatment procedure; or
implementing changes as per long-term plans or higher-level policies [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ].
3.2
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>Transition from Stratum III to Stratum IV</title>
        <p>
          At Stratum III level of complexity, employees have a considerable degree of
autonomy and are empowered to take initiative on their own. This allows fast and systematic
response to a large market. However, this differentiation begets decentralization that
sows the seeds of the “crisis of control” [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]: the control over the organization as a
whole is lost. As the organization grows or develops, attempts to return to centralized
management are doomed to fail, while development to the next level calls for
integration and coordination of hitherto siloed domains.
        </p>
        <p>
          Work at Stratum IV is markedly more strategic. It is necessary to think beyond
individual products, services, systems, or units, and to integrate, manage and support
interactions between a number of systems and practices. Whereas at Stratum III work
always has to be done within given concrete resources and limits, at Stratum IV a
broader overview and comprehensive management of the organization are necessary:
budgets and resources can be allocated and shifted in order to align the
comprehensive output of the organization with the needs of the constituents that it serves [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ].
Translation of the different streams of work into financial terms will help see the net
effects of changes in various parts of the organization. Instead of designing and
optimizing an individual Stratum III work system, relevant questions at Stratum IV are
externally focused and start from non-material things such as customer benefit or
values: “What does the customer really need? What are people concerned with
today?” [cf. 15].
3.3
        </p>
      </sec>
      <sec id="sec-3-3">
        <title>Stratum IV</title>
        <p>
          At Stratum IV, the focus shifts away from operational concerns to managing both
continuity and change [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. Comprehensive provision [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] of output entails constant
introduction of new products or services and decommissioning of old ones in order to
reshape profitability and to provide output that is comprehensive enough in terms of
range and coverage. At this level, direct control over the domain of a mutual
recognition unit is no longer possible. Management is less direct and more about coordination
of multiple functions. The focus is on the design and operation of an integrated set of
systems, whose interactions are integrated and controlled [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ].
        </p>
        <p>
          Stratum IV is about breakthrough innovation of new products and services and
discovery of new markets [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. Work at this level requires intuitive judgment to detect
gaps in services and to compare known systems with one another, but not to develop
yet unknown systems [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. Senior executives at this level translate the strategic intent
and demand signals in their larger context into more tangible objectives and concrete
plans for operating units. They must hold together business in the present whilst at the
same time building for the future [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ].
3.4
        </p>
      </sec>
      <sec id="sec-3-4">
        <title>Transition from Stratum IV to Stratum V</title>
        <p>
          At Stratum IV, coordinative means such as organization-wide programs, federated
governance mechanisms and profit sharing schemes allow the organization’s limited
resources to be allocated effectively. However, at some point the many systems and
programs tend to outgrow their original intention and become overly bureaucratic.
The ensuing “red-tape crisis” [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ] needs to be resolved through less formal, normative
control and interpersonal collaboration of Stratum V.
        </p>
        <p>
          A move from Stratum IV to Stratum V is marked by a much more open definition
of the product field: concrete terms like “kitchen chairs”, “microscopes” or
“telephones” at Stratum IV will be replaced by broader terms like “furniture”, “scientific
instruments” or “communications equipment” [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ]. Within this broad description, there
is no precise picture of what is required to provide the product or service; the field is
more abstract and open-ended.
3.5
        </p>
      </sec>
      <sec id="sec-3-5">
        <title>Stratum V</title>
        <p>
          Field coverage [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] at Stratum V expands the scope from a range of products or
services to a framework that specifies a general field of need. Changes at this level
pertain to entire ranges of products and services, involve long-term strategies and entail
social, political, and financial considerations. Stratum V is the first level where
fullscale business units or businesses – unified whole systems – are elementary entities
[
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]. It is about creating new business models [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ] and requires the capacity to redefine
the rules, to change the boundaries of the organization, and to engage in strategy
development [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. The organization’s current and potential future role within the
business environment as well as the influence of social, political, economic and
technological factors must be understood.
4
        </p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Three Logics of Product Development</title>
      <p>
        Product markets differ in terms of stability of technologies and customer preferences
[
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. Different competitive contexts give rise to intrinsically different strategy
concepts, product strategies, and product creation processes. Sanchez [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] suggests a
typology of three increasingly more dynamic product market contexts: stable, evolving,
and dynamic. Respectively, Sanchez and Mahoney [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] distinguish three approaches
to product design: sequential, overlapping, and modular.
      </p>
      <p>
        In stable product markets [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], technologies and market preferences are stable, and
strategic management pertains to strategic commitments, control of production
processes, vertical integration, and defense of competitive position. Product strategies
focus on increasing market share by reducing costs for producing standard products
and by extending control of distribution channels. Product differentiation is largely
limited to non-product dimensions, such as service or advertising.
      </p>
      <p>
        In the respective “traditional” sequential development process [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ], the
technological development and specification of interdependent product components is
sequential and at most episodic. Information flows from one development stage to the next.
There is no overlap of development processes, but feedback from one development
stage to prior stages is possible. The sequential process is subject to breakdowns,
losses, and delays. This approach requires a tightly coupled organization structure: a
single organization or vertically integrated entities. Product architecture is the output
of the design and development process.
      </p>
      <p>
        Evolving product markets [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] entail technological change or evolving preferences
of customers. Strategy concepts focus on strategic adaptation to change, on bundling
relevant resources to the new competitive conditions, and on re-engineering business
processes. Product differentiation by features and performance increases in
importance. Adoption of new technologies, introduction of new products, and
development of new product features are optimized vis-à-vis the changing market conditions.
      </p>
      <p>
        These new capabilities call for collaboration with complementors and overlapping
problem solving approach that organizes the sequential development into overlapping
stages [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. This approach improves information flows between development tasks,
speeds up component development, and reduces information losses between stages.
The overlapping problem solving process has evolving product architecture and
requires intensive managerial coordination of incompletely specified development
tasks. The organization structure in this approach is often team-based.
      </p>
      <p>
        Dynamic product markets [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] are characterized by accelerated evolution of product
concepts, manufacturing process capabilities, and product coordination technologies,
as well as much more varied and demanding customer preferences. The focus of
strategic management shifts from “managing strategic change” to a “higher order”
process of rapidly reconfiguring the organization to changing circumstances on an
ongoing basis.
      </p>
      <p>
        The respective “modular” organization of product development processes [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]
assumes fully specified component interfaces of a modular product architecture before
beginning development of components. The stable information structure of a fully
specified product architecture helps avoid breakdowns, losses, and delays in
information flows. The organization is intentionally decomposed to loosely coupled,
coordinated and flexible network structure.
5
      </p>
    </sec>
    <sec id="sec-5">
      <title>Scaling Agile and Scaled Agile Framework</title>
      <p>
        The term agile was first introduced in the context of software development. Agile
software development promotes self-organization, close collaboration, rapid delivery
of useful software, and adaptation to changing requirements [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. However, these
agile principles are not adequately applicable beyond the team level [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. Leffingwell
[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] considers agile software development methods such as XP [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] and Scrum [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ] as
“software instances of lean,” whereas lean provides a broader framework for
software-based new product development. Leffingwell [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] builds on the framework for
lean software development by Larman and Vodde [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] and the “second generation
lean product development” as outlined by Reinertsen [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. The SAFe framework is
based on Leffingwell [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. Other frameworks of scaled agile include LeSS (Large
Scale Scrum) [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ] and DAD (Disciplined Agile Delivery) [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ].
      </p>
      <p>
        Leffingwell [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] outlines an “agile enterprise big picture” that distinguishes three
levels: the team level, the program level, and the portfolio level. Within a larger
enterprise, there are typically pods of agile teams of about 50 to 100 people each,
organized around building a larger feature, system, or subsystem that constitutes the
program. For a really large system, a number of such programs account for the portfolio.
      </p>
      <p>
        At the team level, agile teams define, build, and test user stories in a series of
iterations and releases. The teams of 7±2 team members are self-organizing with respect to
the work in the program backlog. They are also self-contained, having all the roles
necessary to develop the software features or components the team is tasked to
deliver. Typical roles in an agile team include a product owner, a Scrum Master,
developers, and testers. The team may also include (or share) specialty resources such as
database administrators, user experience experts, or test automation experts, as
necessary to define, develop, test, and deliver working and tested software. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        At the program level, multiple teams synchronize their development in an agile
release train (ART), which produces potentially shippable increments (PSIs) at typically
fixed 60- to 120-day intervals for customer preview, internal review, and system-level
quality assurance. Typically a PSI consists of four to five development iterations
followed by a hardening iteration (with an empty backlog) that is used to resolve defects,
refactor code, and to provide time for release validation and testing. SAFe is based on
the tenet of “develop on cadence, deliver on demand,” allowing the development team
to continuously build incremental product functionality, while marketing/distribution
is free to deploy external releases as necessary. [
        <xref ref-type="bibr" rid="ref5 ref6">5,6</xref>
        ].
      </p>
      <p>
        At the portfolio level, a mix of investment themes establishes investment priorities
for the organization. These themes ensure that the work will be in line with the
business strategy. They drive the portfolio vision that is translated to epic-scale initiatives
that are prioritized, estimated, and maintained in the portfolio backlog. The epics span
several releases and are described at the level of detail that is only sufficient to initiate
a further discussion. Prior to release planning, these epics are converted into more
detailed stories that are allocated to various release trains for implementation. On the
other hand, architectural runway addresses architectural epics, which enable the agile
enterprise to implement high-priority features in short term without excessive,
delayinducing refactoring. [
        <xref ref-type="bibr" rid="ref5 ref6">5,6</xref>
        ].
6
      </p>
    </sec>
    <sec id="sec-6">
      <title>Agile Product Development Maturity Framework</title>
      <p>Based on the theoretical background presented in Sections 3 through 5, we construct a
preliminary Agile Product Development Maturity Framework (APDMF) that is
intended to help assess the product development capability of a given organization and
to inform how the capability can be further developed. As exhibited in Table 1, the
framework integrates together requisite strata, types of product market, the respective
development processes, and the levels in SAFe that help implement the respective
levels of capability. The APDMF outlines three levels of product development
maturity: Linear, Agile, and Coevolutionary.</p>
      <p>
        Linear product development is of Stratum III complexity [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]: serving
systematically, reliably and efficiently a stable product market [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. The development process is
sequential [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]: information is passed from one functional team to the next as the
linear process unfolds. Agile methods, if any, are applied at the team level or at the
team of teams (i.e. program [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]) level.
      </p>
      <p>
        Agile product development addresses Stratum IV complexity [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]: breakthrough
innovation of new products and services to address any current or future value
deficiencies in order to provide output that is comprehensive in range and coverage [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. It is
requisite in an evolving product market [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], in which new technologies are adopted,
new products introduced, and resources reassembled to enact changes in that
comprehensive provision. Product development follows an overlapping logic, in which
informed governance and evolving product architecture enable interleaving of product
development tasks. The agile approach is extended to the portfolio level to govern the
evolution in the product/service mix. An architectural runway [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] or alike makes the
product architecture visible, communicable, and amenable to change.
      </p>
      <p>
        Coevolutionary product development is of Stratum V complexity [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]: ongoing
shifting of the organization’s value proposition and respective transformation of the
business model vis-à-vis the dynamic product market [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. Co-specialized constituents
of the business ecosystem specify and co-evolve a modular product architecture,
whose stable information structure enables a loosely coupled network structure [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].
      </p>
    </sec>
    <sec id="sec-7">
      <title>Implementation of Agile Product Development at RAY</title>
      <sec id="sec-7-1">
        <title>Brief History of RAY’s Product Development</title>
        <p>Product development at RAY dates back to the 1960’s, when RAY developed its
mechanical payazzo games. Slot machines followed in the late 1970’s. Technological
know-how pertained to design of games, production of machines, as well as resource
planning of distribution and maintenance.</p>
        <p>Product development based on software started in 1978, when RAY took on the
task of creating a fruit game type of a slot machine. The development of games and
supportive software increased in the 1990’s, when the machines were connected to the
network and when the first multigame machines were built.</p>
        <p>All in all, it was pretty much business as usual for RAY for almost 70 years:
deploying new slot machines in the field, running casino type table games in night clubs
and Casino Helsinki, collecting money, and channeling it to beneficiary organizations.
The saturation point of the installed base had been achieved and business was not
developing. The mode of operation was largely offline. As building a network for the
slot machines started in 1995, Internet connections were poor, the network was only
used as an incident and reporting channel, and games were operated with coins.</p>
        <p>RAY’s agile game and service development dates back to six years ago. The
programming of games has always been in RAY’s own hands, but only in the last three
years in-house service development has been strongly adopted. This is in line with the
strategy of building and maintaining products in RAY’s own teams, reinforced by
external consultants, rather than sourcing these strategic capabilities from vendors.
7.2</p>
      </sec>
      <sec id="sec-7-2">
        <title>Adoption of the SAFe Framework</title>
        <p>In 2009, RAY took up a new project portfolio tool, whose purpose was to provide a
comprehensive view of the projects in progress at RAY. However, project portfolio
management at this point represented traditional project work, and RAY desired to
work in a leaner manner. As the development of decision-making structures begun
enterprise-wide in late 2013, product development seized the auspicious moment and
adopted selected features of SAFe in 2014. These features constituted the planning
and steering model of RAY’s product development.</p>
        <p>The framework was first adopted in product development, but since then it has
expanded step by step a team or a unit at a time to embrace parties directly connected to
product development. RAY applies the model critically, adopting only practices that
are considered as value-adding to its own work.</p>
        <p>At the time of writing, the management model is as follows:
• The board of directors is responsible for creating strategy.
• The business steering group is responsible for developing and implementing
strategic plans.
• Distribution channels are responsible for implementing channel-specific plans.
• The business operations group is responsible for the coordination of operational
cross-channel work, e.g. pertaining to the SAFe-based “development train” of
product development.
• The development train implements prioritized tasks in both channel-specific and
cross-channel backlogs and reports of the progress to the business operations group
and thereby to the business steering group.</p>
        <p>The governance of agile product development at RAY is illustrated in Fig. 1. All
operational units, including product development, participate in the business planning
process for strategy implementation. In this process, the financial goals for the
coming few years are attained through product and service development initiatives driven
by business needs. Business needs and concept ideas are compared with architectural
needs as described in the enterprise architecture so that the necessary new capabilities
and developmental requirements are recognized as early as possible.</p>
        <p>The concept ideas chosen in business planning end up to the portfolio management
process, in which the concepts are further developed, until some of them are mature
enough to be implemented. A part of the concepts are still translated to projects, but it
only has relevance from a project portfolio point of view. At the time of writing, the
concepting phase is still on the development agenda of product development
management. It might be ideal that within strategic themes, a number of concepts and
project ideas would be cultivated and the best ones be chosen for further
development. The coordination of work selected for implementation as well as dependence
and resource management take place in the so-called planning process of product
development trains.</p>
        <p>As per the SAFe definition, a train is a team of teams, which at RAY involves
around 120 people. Its purpose is to co-develop products and services for business.
The product development train unites the teams in a shared rhythm, in which
planning, implementation and continuous process development occur. The product
development trains are designed to be 10 weeks in duration, so that visibility into future
work is as realistic as possible and that the teams can agree upon the schedule for
common work. In other words, the product development train arranges the work
under one planning umbrella that defines the beginning, end, and quality, but not the
scope. Each team is responsible for the scope, schedule, and releasing within its own
area of responsibility. Each team contributes to the mutual plan by publishing which
tasks are its own and which ones are shared with other teams during the
trainplanning period.</p>
        <p>The work is done in teams, whose backlog consists of 1) maintenance tasks
pertaining to products and services in their area of responsibility and of 2) new
development and testing. The teams are built based on the tenet that they are self-sufficient
and self-governing. Members of the team include the product owner and other
members that can often take on different roles within the team on an equal basis.
7.3</p>
      </sec>
      <sec id="sec-7-3">
        <title>Analysis of Transformation in Product Development Using APDMF</title>
        <p>It is our interpretation that, in the last decade, RAY’s product development has
developed from Linear to Agile, in terms of APDMF levels (see Table 2). This denotes a
developmental transition from Stratum III to Stratum IV complexity, wherein the
specialization of functions and operations has been counterbalanced by respective
integration of these faculties. As RAY started to develop digital services alongside
games, business planning was still domain-specific. Characteristically to Stratum III
operational logic, the organization was siloed in separate and poorly integrated units
of ICT, games development, and service development. Projects were aligned with
strategy, but they focused on single distribution channels, single products, or single
concepts. Decisions on products were made independently of each other within
separate channels. No need was felt to consider the customer perspective across the
channels. Cross-departmental product development projects were rare.</p>
        <p>In the 2010’s, RAY has started to ask itself what its customers truly want. A
watershed in the transition towards Stratum IV was in 2009, when the strategy was revised
and several major initiatives, such as the online casino and the preferred customer
program, were launched. These new capabilities posed a great challenge to RAY’s
organization and technology platform. The programs were independently budgeted,
and for the first time agile methods were employed in implementation teams.</p>
        <p>Internet gaming and the preferred customer program entailed a multi-channel
approach: the business wanted to have same products both online and in physical
locations, and information on the customer was naturally of common interest.
Coordination across teams and units posed great challenges to RAY’s organization, which were
met with agile methods and team organization around different capabilities. In
addition to carrying out the afore-mentioned strategic programs, RAY designed and
started to implement a common cross-channel platform that provides functionality
required by one or more distribution channels as service.</p>
        <p>Over time, it became apparent that the new Stratum IV logic of cross-channel
integration and coherence called for closer connection and better visibility between the
teams and the top management. To enable decentralization of governance and
interfunctional coordination, the organization revised its management model and
implemented the SAFe framework. These organization-wide frameworks provides a
casein-point example of overarching strategic systems that help the organization allocate
its limited resources in the face of Stratum IV complexity.</p>
        <p>Today, product decisions are made in collaboration so that channels, product
managers and product development all have a say before making the decision. This
approach is driven by new overall holistic thinking but is also mandated by the games
that are published in several channels.</p>
        <p>The increase in work complexity has transpired in step with change in RAY’s
product market context. In the pre-Internet era, RAY was a true monopoly in its
legally decreed field. While it is still the only operator of physical slot machines in
Finland, the monopoly is challenged by the proliferation of online casino games and
other digital entertainment that are available to consumers worldwide. The customers’
preferences are more fluid and technological change is more rapid than before. As a
result, quicker responses are required to adjust RAY’s comprehensive provision.</p>
        <p>The development process has evolved accordingly. When business was still
project-based, systems were developed by vendors on a one-off basis, resulting in
suboptimized, siloed solutions. Architecture was an emergent outcome and not planned
from product portfolio point of view. Today, enterprise architecture embraces all
systems that have both a business owner and a technical owner, and products are
designed with respect to the portfolio and its objectives. A product owner plays an
important role in the development of critical systems. He/she is responsible for
developing the product throughout its life cycle and has a team of in-house developers and
testers as well as external consultants, if needed.</p>
        <p>Finally, the scope of agile has grown. In the past, teams were responsible for
systems underlying RAY’s machines. Integrations between these systems were based on
configuration-, reporting- or fault information, but product specifications did not span
across systems. The adoption of agile methods started in these individual teams.
However, they soon formed enclaves of high-quality, agile software development that
stood out from the surrounding organization. At some point, the teams begun to have
increasing demands towards the rest of the organization, which had become the
bottleneck for further increases in quality and performance: strategic development,
concept development and prioritization took too long and were often of inadequate
quality to high-performing agile teams. This prompted the adoption of SAFe at RAY.
Nowadays, product or service development increasingly entails inclusion of several
systems. This calls for inter-team coordination, portfolio management, and
governance.
In the early 2000’s, after decades of relatively stable and predictable business as
usual, Finland’s Slot Machine Association (RAY) increasingly started to sense the
disruptive forces of digitalization in the gaming business: overseas online casinos as well
as changes in consumers’ preferences, use of time, and playing behavior challenged
the status quo and forced RAY to respond by upping its ante (pun intended). In just a
few years, the organization invested in new technology; developed requisite
organizational capabilities to match new requirements of 24/7 availability, information
security, and agility; and managed a transition from a steady operator to an agile innovator.
Seen against the theoretical backdrop of Levels of Work [e.g. 1,2,10], this can be seen
as a shift from Stratum III to Stratum IV complexity and capability.</p>
        <p>
          We surmise that SAFe provides a “scaffolding” that helps the organization develop
its software-based products and services in an agile and lean way. It enables scaling
agile product development to the team-of-teams (i.e. program) level and further to the
portfolio level [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ], which would correspond to Stratum III and Stratum IV complexity
of work [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ], respectively. In the context of product development, a Stratum IV
response would allow the organization to address the challenges of evolving product
markets [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ] and to enable overlapping problem solving logic [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ] across closely
interrelated component design and development tasks. The organization must continually
align its wall-to-wall comprehensive coverage [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] in its product market in the face of
the ever-changing environment, where teams find new solutions and the organization
learns from the customer. Portfolio management requires intensive coordination at all
levels to pace different strands of development in relation to one another in resourcing
and in time. At the portfolio level, the architectural runway of SAFe would support
evolving product architecture, while at the program level agile release trains would
provide a means to synchronize the development efforts of multiple teams.
        </p>
        <p>With its adaptation of SAFe, RAY is well geared to establish continuous, agile
product development in the future. Further work is needed, particularly in developing
the concept development process and in fulfilling the customer needs. Too many
layers still exist between the customer and the product development team, which inhibits
a more direct feedback loop between the two.</p>
        <p>It is to be noted that the proposed Agile Product Development Maturity Framework
(APDMF) is based on theoretical contemplation and only tentatively tested trough this
case study. As such, it will need more empirical corroboration and further theory
building. We view that the framework is not only of theoretical interest but bears a
number of practical implications. The framework would provide a yardstick that helps
managers and other practitioners assess and address capability requirements in order
to further develop the organization’s product/service development vis-à-vis the
organization’s product market. If developed further, it has potential to inform the
development of product development capacity by providing insights into the types of
competencies, systems, structures, and respective investments that will be needed at a given
stage of development. For instance, it can be argued that a dynamic product market
would call for a modular approach, which, in turn, would require Stratum V product
development capability. In the case of RAY, this will not be required in the
foreseeable future, but if it ever will be, insights into work complexity and requisite
capabilities will be helpful in informing how to navigate the organization to the next level of
complexity and coherence.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Jaques</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          (
          <year>1998</year>
          ).
          <article-title>Requisite Organization: A Total System for Effective Managerial Organization and Managerial Leadership for the 21st Century, Revised second edition</article-title>
          . Baltimore, MD: Cason Hall &amp; Co.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Rowbottom</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Billis</surname>
            ,
            <given-names>D</given-names>
          </string-name>
          , (
          <year>1987</year>
          ).
          <article-title>Organisational Design: The Work-Levels Approach</article-title>
          . Aldershot, UK: Gower.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Sanchez</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>1996</year>
          ).
          <article-title>“Strategic Product Creation: Managing New Interactions of Technology, Markets, and</article-title>
          <string-name>
            <surname>Organizations.</surname>
          </string-name>
          ”
          <source>European Management Journal</source>
          ,
          <volume>14</volume>
          ,
          <issue>2</issue>
          ,
          <fpage>121</fpage>
          -
          <lpage>138</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Menor</surname>
            ,
            <given-names>L. J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Roth</surname>
            , Al.
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>“New Service Development Competence and Performance: An Empirical Investigation in Retail Banking</article-title>
          .
          <source>” Production and Operations Management</source>
          ,
          <volume>17</volume>
          ,
          <issue>3</issue>
          ,
          <fpage>267</fpage>
          -
          <lpage>284</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Leffingwell</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise</article-title>
          . Addison-Wesley.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>SAFe</surname>
          </string-name>
          (
          <year>2014</year>
          ).
          <source>SAFe Version 3</source>
          .0,
          <string-name>
            <surname>July</surname>
            <given-names>28</given-names>
          </string-name>
          ,
          <year>2014</year>
          . Retrieved from http://www.scaledagileframework.com/
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Bendoly</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bharadwaj</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Bharadwaj</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2012</year>
          ). “Complementary Drivers of New Product Development Performance:
          <string-name>
            <surname>Cross-Functional</surname>
            <given-names>Coordination</given-names>
          </string-name>
          ,
          <source>Information System Capability, and Intelligence Quality.” Production and Operations Management</source>
          ,
          <volume>21</volume>
          ,
          <issue>4</issue>
          ,
          <fpage>653</fpage>
          -
          <lpage>667</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>McMorland</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>“Are you big enough for your job? Is your job big enough for you? Exploring Levels of Work in organisations</article-title>
          ,” University of Auckland Business Review,
          <volume>7</volume>
          ,
          <issue>2</issue>
          ,
          <fpage>75</fpage>
          -
          <lpage>83</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Connor</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          and Mackenzie,
          <string-name>
            <surname>P.</surname>
          </string-name>
          (
          <year>2003</year>
          ). “
          <article-title>The leadership jigsaw - finding the missing piece</article-title>
          ,
          <source>” Business Strategy Review</source>
          ,
          <volume>14</volume>
          ,
          <issue>1</issue>
          ,
          <fpage>59</fpage>
          -
          <lpage>66</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Macdonald</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Burke</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Stewart</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          (
          <year>2006</year>
          ).
          <article-title>Systems Leadership: Creating Positive Organizations</article-title>
          . Aldershot, UK: Gower.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Kinston</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Rowbottom</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>1989</year>
          ).
          <article-title>“Levels of work: New application to management in large organisations</article-title>
          ,
          <source>” Journal of Applied Systems Analysis</source>
          ,
          <volume>16</volume>
          ,
          <fpage>19</fpage>
          -
          <lpage>34</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Van Clieaf</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Kelly</surname>
            ,
            <given-names>J. L.</given-names>
          </string-name>
          (
          <year>2007</year>
          ).
          <article-title>“The New DNA of Corporate Governance</article-title>
          .” In K. Shepard,
          <string-name>
            <given-names>J. L.</given-names>
            <surname>Gray</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. G.</given-names>
            <surname>Hunt</surname>
          </string-name>
          , and S.
          <source>McArthur (Eds.)</source>
          , Organization Design, Levels of Work &amp; Human
          <string-name>
            <surname>Capability</surname>
          </string-name>
          (pp.
          <fpage>103</fpage>
          -
          <lpage>114</lpage>
          ). Global Organization Design Society,
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Gould</surname>
            ,
            <given-names>D. P.</given-names>
          </string-name>
          (
          <year>1986</year>
          ).
          <article-title>“Stratified systems theory in the design of organization-wide information systems</article-title>
          ,”
          <source>International Journal of Information Management</source>
          ,
          <volume>6</volume>
          ,
          <issue>1</issue>
          ,
          <fpage>5</fpage>
          -
          <lpage>15</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Greiner</surname>
            ,
            <given-names>L.E.</given-names>
          </string-name>
          ,
          <year>1998</year>
          [1972].
          <article-title>“Evolution and revolution as organizations grow</article-title>
          .” Harvard Business Review, May-June 1998 [
          <article-title>originally published in the July-August 1972 issue]</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Glasl</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          (
          <year>1997</year>
          ).
          <article-title>The Enterprise of the Future</article-title>
          . Hawthorn Press.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Sanchez</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Mahoney</surname>
            ,
            <given-names>J. T.</given-names>
          </string-name>
          (
          <year>1996</year>
          ). “Modularity, Flexibility, and
          <article-title>Knowledge Management in Product and Organization Design</article-title>
          .”
          <source>Strategic Management Journal</source>
          ,
          <volume>17</volume>
          ,
          <fpage>63</fpage>
          -
          <lpage>76</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Agile Manifesto</surname>
          </string-name>
          (
          <year>2001</year>
          ).
          <article-title>Manifesto for Agile Software Development</article-title>
          . Retrieved from http://agilemanifesto.org/.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Laanti</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>“Characteristics and Principles of Scaled Agile</article-title>
          .” In Dingsøyr et al. (Eds.),
          <source>XP 2014 Workshops, LNBIP 199</source>
          ,
          <fpage>9</fpage>
          -
          <lpage>20</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Beck</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          (
          <year>2000</year>
          ).
          <article-title>Extreme Programming Explained: Embrace Change</article-title>
          . Boston, MA: Addison-Wesley.
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Schwaber</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>Agile Project Management with Scrum</article-title>
          . Redmond, WA: Microsoft Press.
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Larman</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Vodde</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Scaling Lean &amp; Agile Development: Thinking and Organizational Tools for Large-Scale Scrum</article-title>
          . Addison-Wesley.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Reinertsen</surname>
            ,
            <given-names>D. G.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>The Principles of Product Development Flow: Second Generation Lean Product Development</article-title>
          . Redondo Beach, CA: Celeritas Publishing.
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Larman</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Vodde</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Large-Scale Scrum: More with LeSS</article-title>
          .
          <string-name>
            <surname>Addison-Wesley Professional</surname>
          </string-name>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Fowler</surname>
            ,
            <given-names>S. W.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Lines</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2012</year>
          ).
          <article-title>Disciplined Agile Delivery: A Practitioner's Guide to Agile Software Delivery in the Enterprise</article-title>
          . IBM Press.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>