The replacement of legacy systems in government agencies is a challenging and complex process. Legacy software features and the business practices they support tend to be reproduced in replacement applications, because practitioners are averse to the risks associated with business process change. As a result, opportunities to introduce efficiencies and innovation with new technologies are missed. Since legacy replication occurs prominently during the requirements phase of legacy replacement projects, an online role-playing game (PROVO) was designed to promote creativity during the analysis and critique of business requirements stemming from legacy systems. The game was evaluated by a small group of practitioners from a municipal organization in the United States. The findings of the evaluation session are discussed and used to inform changes to PROVO's design.
In the last couple of decades, games have been applied successfully in education,
workforce training, for military simulations, scientific exploration and other
nonentertainment purposes [
The area of information technology projects involving legacy system replacement is
one such complex area. When replacing outdated technologies and applications (e.g.
mainframe systems, heavy client-based software that is not Web-enabled, or any
application built on an aging platform that cannot be easily expanded or maintained)
practitioners are often faced with the dilemma to ensure stability in the process of
transition, while simultaneously facilitating technological and business innovation.
This dilemma can be felt most ostensibly at the requirements phase in projects where
software replacing a legacy system is either being developed or acquired. It is at the
requirements juncture when business experts must make decisions regarding which of
the old system’s features are essential enough to carry over into the new system, and
which can be phased out or replaced. Innovation, however, often gives way to
riskaverse, conservative attitudes in the public sector, which tend to preserve the status
quo [
This paper is organized as follows. The motivation for our research and an overview of the legacy problem in government agencies are presented in the subsequent section. Our proposed approach - the development and evaluation of a game, is introduced in Section 3. The game’s design and theoretical foundations are described in Section 4. Section 5 presents the outcomes of an evaluation of the game’s core concepts and flow by practitioners, and Section 6 offers a discussion of the findings. The last section outlines some conclusions and directions for future research work. 2
In government agencies legacy systems are often seen as a source of stability and the
operations they support are frequently deemed too critical to make changes to [
We argue that the legacy problem and the propensity towards risk-aversion in public organizations reveal the need for a participative tool which deals with requirements activities critically and creatively. Such a tool would assist in revealing the true historical sources of certain requirements and debunking their operational immutability by explicitly juxtaposing perspectives related to stability and risk avoidance to those promoting business process improvement and innovation. To this end, we envision an online game to address the legacy problem, in which practitioners can critique business requirements for being either too risky or not innovative enough: the game context should free up the participants’ creative thinking, as it is assumed that the game is an exercise the outcomes of which do not have to materialize in an actual project, thus promoting the consideration of more radical requirements changes.
We also argue that triadic game design can inform our game’s development as it
distinguishes between three main areas of design – the ludic (play), semiotic
(meaning), and the ontological (reality) [
The ludic aspect refers to the techniques by which a game is made interactive,
challenging, fun and immersive. The key technique we propose to employ is
roleplay. Since games create an artificial, fictional setting, participants often assume
different roles, characters, or personas, that allow them to explore a diverse set of
behaviors, assumptions and take symbolic actions, partly as a result of the
phenomenon of the “online dis-inhibition effect”, according to which in online games
people do things they would not otherwise do in the physical world [
The ontological aspects of a game encompass the underlying model of the real-world domain the game is based on. In our case, the reality represented is that of requirements analysis activities during a legacy system replacement project. Actual requirements from an active project will be used in the game, and thus its ontological grounding will be more robust.
We follow a design and creation methodology [
The main premise of our game is that of the requirements challenge - one player questions a particular requirement, and others must justify it. Therefore, we have called our game PROVO, which in the international language Esperanto means test or challenge.
The semiotic and ontological aspects of our game are based on Colin Potts et al.’s inquiry cycle model, and the game’s roles are inspired by Neil Maiden’s creativity workshops.
In their model for the support of discussions about system requirements, Potts et al.
[
Just like Potts et al.’s approach, PROVO aims to direct exploratory thought during the
requirements phase [
Another goal of the creativity workshops is to open up the solutions space by encouraging out-of-the-box thinking, and also to enable participants to communicate their opinions more freely, by allowing them to voice any frustration or dissatisfaction they might be experiencing. Similarly to such workshops, our game provides a “safe space” for competition and opposition - it is well accepted if a person is competitive during gameplay, whereas in any other workplace situation conflict is generally avoided.
PROVO’s initial design is based on the actions of two main players – the innovator and the heritage keeper, who both issue challenges towards specific business requirements. The innovator must make sure the new system simplifies and improves business processes, and the heritage keeper is tasked with ensuring that the new system does not introduce instability and other operational risks. The requirements are derived from a real legacy system replacement project taking place in the participants’ government agency, hence the players are familiar with the domain and the system which is at the center of the game is not hypothetical. The requirements are listed for everyone to see. The innovator identifies those requirements which replicate the old system’s features unnecessarily, and issues individual challenges to each such requirement stating the reasons for the challenge. The heritage keeper must also issue challenges – he, or she can state that a particular requirement introduces too much of a departure from the status quo and this would introduce business risk. Additional players in the game are the problem solvers, who respond to these challenges by either morphing a requirement to meet a challenge of their choosing, or by providing justifications as to why a requirement should remain unchanged. Another player – in the role of arbiter – decides if a challenge is met, and moves the requirement to either the Innovation, or the Heritage space, where it is no longer subject to change. There is also Neutral space for requirements which no one wishes to challenge, and which are not controversial. The game ends when all requirements have been assigned to a space, and whether the Innovator or the heritage keeper wins is determined from the number of requirements they manage to get assigned to their respective space. The roles would be assigned by the game software itself, after the participants answer personality questions, which assess their propensity towards novelty, their resistance to change and other traits relevant to the problems of risk aversion and legacy preservation in organizations.
Players are able to participate anonymously and asynchronously in PROVO. The
online medium makes anonymity possible, and this is beneficial where workplace
hierarchies or other factors tend to stifle open discussion [
PROVO supports creativity by enabling individuals to participate in a role playing
exercise with well-defined rules. According to the structuralist approach [
A low fidelity try-out of the game was conducted with a team of non-technical business experts at a municipal government agency in the US. The purpose was to evaluate PROVO’s basic roles, rules and dynamics for their adequacy in discussing requirements in legacy system replacement projects. The evaluation session was intended to imitate the flow of the game using a generic online communications tool, without a specifically developed graphical user-interface.
The tryout involved 5 participants – the heritage keeper, the innovator and three problem solvers. The innovator and the heritage keeper would issue challenges to 10 sample requirements for a new Web portal applications for citizens, which replaces the old municipal website. The arbiter wasn’t assigned as part of the try-out intentionally. Observing the natural propensities of the players without the arbiter’s intervention was part of the game concept assessment.
The players were gathered beforehand and presented with the basic rules and concepts. The roles were assigned randomly. The problem solvers’ role was to respond to challenges by proposing either mitigations to requirements deemed risky, or modifications to requirements that are not innovative. The game was carried out as a Web conference (a WebeEx meeting) during which each player participated from his or her own office, and typed challenges or responses to challenges in a common chat window, while the 10 sample requirements were displayed as a numbered list in the background on the screen. Each participant would type in the number of the requirement they were referring to and then their suggestions for requirements modification or justification. In the Web conferencing service used, participants had to type in a screen name, and most selected screen names that did not reveal their identity. Only the player in the heritage keeper role chose a screen name that revealed his identity.
The Web conference session was programmed for 45 minutes. After it ended it was discovered that due to technical issues the player in the innovator role did not log in and participate. None of the other participants were aware, due to the option for anonymous participation in the session. The problem solvers were essentially responding only to the challenges of the heritage keeper and suggesting changes to the listed requirements. The player who was in the heritage keeper role gave general feedback on the requirements, rather than address risk specifically and also made some suggestions on morphing the requirements. All players participated actively and expressed their opinions about each of the 10 requirements. A transcript of the session was preserved.
During the evaluation, it became evident that the players may not follow their prescribed roles accurately, and they may make broad and diverse comments which are not necessarily aligned to innovation or risk issues. The arbiter role would be important in this context, as this role is responsible for adherence to the game’s rules. Therefore in future evaluations of the game an individual (someone other than the game’s designers) must be assigned as an Arbiter and provided instructions on fulfilling the role. It may also be possible to implement automatic mechanisms to ensure observance of certain rules in subsequent design iterations.
Another observation from the try-out is that asynchronous mode of play (i.e. allowing players to log in whenever they want within a predetermined general time span such as days or weeks) may be problematic, because the competitive dynamics may not emerge if some players (more specifically the heritage keeper, the innovator or the arbiter) are absent to respond to or to engender actions. During the evaluation session, the Innovator did not log in for the game’s duration, and in that timeframe a lot of comments and suggestions were made. The problem solvers may have made different comments, had the Innovator made their challenges before the heritage keeper, for instance.
Furthermore, the importance of a visual structure in the exchange of comments was underscored. If the participants had done this exercise within a designed game interface, and not a plain chat window, it would have been easy to identify that the innovator’s input was missing, and threads would have formed visually for each requirement, thus better enabling a structured dialog. Also, since there were no visual cues such as approvals, rankings and/or “thumbs up” for the comments of the problem solvers, the element of competition was downplayed, and the feedback ended up being somewhat general and non-interactive.
With the current set of roles a minimum of 6 players are needed to ensure adequate role distribution, and such a level of participation may be difficult to achieve at all times. The heritage keeper and Innovator roles currently possess significant control over the game’s flow, as without their challenges a requirement may not become subject to discussion. Also, a sufficient number of problem solvers might be impossible to find, as the individuals in this role must be both objective and knowledgeable enough to propose modifications and justifications without bias. For this reason, adjustments to the game’s role model would be beneficial. 7
The issue of unnecessary replication of old business processes arises when government agencies plan for the replacement of their legacy systems. It is difficult for practitioners to depart from old system and business models, because they fear that such a departure is too risky for operations, or they cannot disentangle requirements that are essential from those that stem from the old system’s limitations. The concepts of an online game (PROVO) were developed to help practitioners discuss and reformulate requirements in legacy system replacement projects. The roles and game flow were designed to promote creativity by allowing participants to argue, and challenge each other in a positive and playful manner.
A pilot session with a conceptual prototype of PROVO was carried out with a small group of government sector employees. The tryout uncovered the need to refine the game’s design so that roles are more closely adhered to, and creativity is encouraged through a more structured focus on risk and innovation. To achieve this, the user interface must be designed to promote on-topic discussion, the role model must be simplified, and role assignment must be automated to make participant recruitment and game execution more practical. In addition, the issue of asynchronous vs. synchronous gameplay arose, which later design iterations must tackle so players can experience the game at their convenience without a loss of focus and momentum. Once PROVO is finalized, its utility must be put to the test by evaluating it in the context of real legacy replacement projects, rather than with hypothetical scenarios. Such rigorous evaluation will promote a better understanding of how creativity can be fostered during the requirements phase of critical IT projects in the public sector, such as those for the replacement of legacy systems.