RE-PROVO: An Evaluation of Gamification in a Law Enforcement Organization

Abstract

Government organizations rely extensively on legacy systems for their operations. When such systems are phased out, the new applications which replace them often replicate legacy functionality unnecessarily, resulting in inefficiencies and missed opportunities for innovation. A prototype of an online discussion game designed to promote the analysis and critique of functional requirements for legacy system replacement and encourage creativity, was evaluated in a local law enforcement agency. The preliminary findings of the evaluation are discussed, and the potential effects of gamification on the future of organizational communications and decision-making are considered.

Keywords

• eol>Requirements Engineering
• Legacy Systems
• Gamification
• Bureaucracy
• Public Sector

Introduction: The Legacy Problem in

Government Agencies

Government agencies are traditionally associated with bureaucracy, inertia and outdated information technology (IT) systems [ 1 ]. Legacy technology is rampant in public sector organizations. “Green screen” mainframe applications, and non-web based systems are particularly prevalent in law enforcement and public safety institutions [ 2 ]. Many of these systems are in the process of being modernized, or replaced, and such projects are costly and time-consuming. When agencies undertake technology modernization, the new applications which are being implemented often mimic the old legacy systems which they are intended to replace. This occurs for several reasons: existing processes are being recreated so that users do not have to be retrained, old data schemas are being retained and extended for purposes of compatibility with other legacy systems, old features are being preserved to comply with existing legislation [ 3 ]. Functional and data specifications are also replicated because agencies fear that change will lead to operational destabilization, chaos or unintended outcomes both for the internal users of the system, and for the public. The phenomenon of risk aversion, and uncritical acceptance of an organization’s operational/business process status-quo which leads to the de-facto reproduction not only of legacy technology and data models, but also of antiquated organizational work processes, has been defined as the legacy problem of the public sector [ 4 ].

The Requirements Phase of Legacy Replacement

Projects

The legacy problem manifests itself during the requirements phase of legacy replacement projects, when business users and IT staff alike are deriving requirements for either commercial off-the-shelf (COTS) or bespoke systems by drawing directly from legacy systems specifications, usage manuals, or even legacy code (since often the only place business processes and rules are documented is in software code [ 5 ]). This is detrimental because government organizations miss the opportunity to revisit, update and streamline their workflows, business processes and operational practices, and to be innovative.

During the requirements phase in legacy system replacement projects in government agencies, often the requirements for new systems are not even discussed instead they are virtually “mot à mot” (word for word) derived from the legacy system’s features. The project management philosophy in such cases is to move everything to the new technology platform as-is as a first phase, and to consider potential changes and improvements later, as a second phase [ 6 ]. Phase 2, however, rarely occurs due to budget constraints or changes in IT project priorities. In some cases, requirements discussions do in fact take pace, and when they do, proposals for departure from the statusquo are commonly rejected by someone in the organization who takes on the role of “devil’s advocate” and brings up all the potential negative, or catastrophic effects of doing things differently from how they have always been done. On the other extreme, there are agency executives who issue directives to make innovations - i.e. implement new trendy technology, without regard for the actual impact that this may have on operations or on the services provided to the public. These two diametrically opposed positions - the naysayer, who fears all change, and the reckless innovator who fails to perform an impact analysis – correspond to stereotypical attitudes towards risk [ 7 ] and to organizational personas that tend to stifle productive functional requirements analysis and elicitation sessions in organizations and to introduce emotional conflict which endangers constructive deliberation. Additionally, project participants’ locus in organizational hierarchy tends to be an influence on whether their concerns are even voiced in such discussions [ 3 ].

A potential approach towards addressing this issue is to enable public sector practitioners to be creative during the requirements phase, and to explore innovative alternatives in depth when discussing and analyzing business requirements for applications that are meant to replace legacy systems. Transdisciplinary and gamebased approaches have commonly been adapted to address “wicked problems” [ 8 ]. Wicked problems defined as issues of a complex techno-social nature [ 9 ], exhibit characteristics similar to those of the legacy problem – they are intractable, contradictory and have shifting formulations. Gamification can be applied to ensure that during the definition and deliberation of requirements affecting work processes in government organizations, arguments for the innovation of existing workflows and operational procedures are expressed and considered when specifications for new systems are developed.

RE-PROVO - Gamifying Inquiry-Based

Requirements Analysis

We have hypothesized that introducing game elements into requirements discussions - elements such as roles, teams, points, badges, and anonymizing participation, would result in the development of requirements which do not uncritically duplicate the legacy system that is being replaced. Gamification may encourage participants not to “take the path of least resistance” and automatically adopt the “safest” approach, but to suggest changes that take advantage of new technology, and introduce efficiencies. By anonymizing online deliberations, and introducing incentives for players to contradict and argue with current requirement formulations, the game design tries to downplay, or disable influences that tend to constrain discussions and brainstorming in traditional formats (e.g. peer pressure during in-person meetings)[ 10 ]. A requirements game - RE-PROVO (Esperanto for retest), was designed as a gamification layer to the Potts et al. requirement inquiry cycle [ 11 ]. According to Potts and his colleagues when a requirement is initially defined, it must go through a critique – i.e. a “challenge” to its current contents, followed by subsequent analysis and discussion, and resulting in its morphing into a different version. The resulting morphed version can be challenged and reformulated as well, and the cycle can (should) repeat until an improved and agreed-upon version of the requirement is arrived at. We borrowed from the inquiry cycle, and used its “challenge” construct as a game action. Two user/player roles were also established to structure the discussion specifically along the themes of change and status-quo preservation - innovators and heritage keepers. The players in the game are randomly assigned to one of these roles, and two teams are formed. The heritage team must issue heritage, or legacy-preservation challenges, through which it critiques any one of a set of requirements listed in the game application by identifying in them issues that may lead to risk, operational instability, substantial changes to standard operating procedures, or departures from existing policies and legislation. The innovations team must issue innovation challenges, which critique the requirements for being too “faithful” to the status quo, or for replicating inefficient processes, thus not taking advantage of new technology to streamline workflows. After challenges have been issued, the players should morph the requirements, so that the critiques raised in the challenges are addressed. Morphs can also be challenged in their own right, and the discussion cycle for them can be repeated. For each action challenging, morphing or commenting, the players receive points. As the points accumulate, the players can also be awarded different types of badges. After an agreed-on time frame, the players are enabled to vote on the requirements and morphs so that a winning version for each requirement thread is elicited. All the points are tallied by team and by individual, and a winning team and “most valuable player” (MVP) are announced.

JIRA’s issue tracker [ 12 ] (by Atlassian) was customized and extended with a gamification add-on - Jiraffe [13] (by BugPotion), and the resulting functionality was used to evaluate the game concept in sessions with practitioners from a law enforcement government agency. While the evaluations of the RE-PROVO prototype are still ongoing, some preliminary findings have been already singled out at this phase of our research.

Evaluation in Law Enforcement Information

Technology Projects

RE-PROVO was tested in a local police department. The employment of a game for purposes of IT requirements elicitation and development in a public safety organization was rather unusual and a departure from traditional IT project management practices. The use of game dynamics in the discussion and analysis of requirements for new technology features and applications revealed thought-provoking insights into the impact of gamification on organizational communications and decision making. Three main themes emerged: 1) employee engagement in group deliberation and collaborative analysis, 2) the effect of power relations on creativity and innovation, 3) embeddedness of gamification in core operations. Employee Engagement in Group Deliberation and Collaborative Analysis Participants in the RE-PROVO evaluation more readily engaged in online discussions, and they felt that the game elements were interesting to explore.

Employees often have difficulty voicing their opinions regarding how their organization should function - they might feel their suggestions will be disregarded, or they might be afraid of how others will perceive them. And when deliberating on information technology issues in particular, if they are business users of systems - they might feel incompetent or intimidated. Therefore, any additional measures to encourage engagement and discussions and brainstorming would help the organization successfully solicit feedback. Introducing game dynamics tends to promote participation initially, but with repeated uses of gamification, the novelty of format may subside. Umar Ruhi asserts that enterprise gamification design must be meaningful if it is to sustain involvement and result in prolonged interest [ 14 ]. An important question becomes whether gamification will need to become permanently embedded in organizational processes which require enhanced employee engagement and pro-active involvement. Will engagement on behalf of the organization’s workers no longer be assumed to be part of one’s work performance, instead perpetually requiring some sort of incentivization or extrinsic stimuli? While games and tool gamification may have a positive effect on individual projects or work processes, it must be asked whether a hunger for game dynamics in all enterprise workflows or projects is sustainable or desirable.

Effects of Power Relations on Creativity and Innovation RE-PROVO participants felt that anonymity was beneficial, but they were nonetheless interested in finding out who the other players were.

Organizations in the public sector are increasingly facing pressure to be innovative, to do “more with less” and to “think outside of the box” [ 15 ]. Given the legislative constraints and the lack of public trust they often encounter, this is a sufficiently complex challenge. It has become a regular occurrence for employees to be called upon to give ideas, get involved in suggestions to overcome problems, propose creative solutions, and participate in brainstorming sessions. However, as indicated by participants in the RE-PROVO game evaluation, there is apprehension to share opinions and ideas whenever management or agency executives are present. When individuals who are positioned high in the organization’s hierarchy are present in meetings to define systems requirements, they (often unintentionally) stifle discussion. Employees may just echo whatever comments managers make, or they may refrain from showing their disagreement. REPROVO was made anonymous precisely so that power relations do not become a factor in deliberations. The online medium made this possible. During the evaluation, our participants did make attempts to guess or uncover the others’ identities (their screen names were fictitious) by exploring various sections of the application. This suggests that identity and one’s position in the organization are important determinants when evaluating others’ ideas, comments or critiques. In RE-PROVO we wanted the players to assess the requirements, challenges, morphs and comments on their own merit; we also wanted participants not to be afraid to challenge anyone or argue with others. In technical discussions in particular, alternative designs and architectures can be more easily assessed from a purely technical perspective, without reference to additional information such as the background of the person making the suggestion. In this sense, anonymous online discussion tools with gamified elements that promote competitive behaviors and productive conflict, have the potential to subvert traditional open, face-to-face methods that seek to elicit innovations and creative solutions. Will gamification help support participation in organizational innovation by those who are more introverted, or in lower positions in the organizational chart? Perhaps gamified organizational tools that support group deliberation and decision-making can become the “true equalizer” [ 16 ]. This is particularly relevant in law enforcement agencies where chain of command considerations may preempt solution or idea quality. Embeddedness of Gamification in Core Operations The participants in the RE-PROVO requirements game evaluation were interested if their winning morphs would be actually implemented, or if the game is just a simulation of a requirements elicitation process. The theme of operational embeddedness of games in the enterprise refers to the manner in which gamified tools and processes result in the creation of a product, or an actionable item. A significant number of games, or game-based applications primarily affect areas that are ancillary to core operations, i.e. they enable educational activities and training, brainstorming, or employee networking [ 17 ]. There are some examples where games introduce incentives in sales, or customer service performance, or are integrated in a quality assurance process (e.g. employees are encouraged to detect more issues, or software bugs [ 18 ]), and these are indeed the main functions of the company. In our case, the game try-out was undertaken for research purposes, and even though it contained real scenarios and requirements from actual ongoing projects, it was primarily an exercise in deliberation, and its outcomes have no guarantees of impacting the agency’s IT and law enforcement decision makers. RE-PROVO would be, in effect, a rehearsal for future discussions, just as many other games or gamified applications are primarily educational, training tools. This echoes the notion of “procedural rhetoric” introduced by Ian Bogost [ 19 ], which posits that the main impact of games is to imply and teach a certain procedural model of the world. It would be a relevant line of inquiry to determine if gamification can involve more than “procedural rehearsals” of the organization’s core processes, but could be directly integrated in decisionmaking (e.g. versions of systems requirements with the most votes in the RE-PROVO game would automatically become a part of the new system’s specification document). In such a scenario an organization’s decision-making processes would be impacted substantially by game dynamics, and gamified activities will be, in fact, more than “just a game.”

Conclusion

A requirements deliberation game - RE-PROVO, was prototyped to evaluate if elements such as role-play, teams, points and badges can assist practitioners in government organizations to tackle the legacy problem, and facilitate the analysis of functional requirements for the replacement of legacy systems. The evaluation raised important issues related to the role gamification can play in organizational communications and decision making in the workplace of the future. Gamified tools and work processes have the potential to be fully integrated in core production-level processes, and to subvert traditional hierarchical decision-making. In law enforcement agencies, which rely on strict command and control structures, gamification may promote improved organizational agility and lead to more innovative outcomes. 13. http://bugpotion.com/

References

1. Vincent M. F. Homburg . Red Tape and Reforms: Trajectories of Technological and Managerial Reforms in Public Administration . International Journal of Public Administration 31 , no. 7

2. Hsinchun Chena , Jenny Schroeder, Roslin V. Haucka , Linda Ridgeway, Homa Atabakhsh, Harsh Gupta, Chris Boarman, Kevin Rasmussen, Andy W. Clements . 2003 . Coplink Connect: Information and knowledge management for law enforcement . CACM , 46 ( 1 ): 28 - 34 .

3. Assia Alexandrova , Lucia Rapanotti, Ivan Horrocks . 2015 . The legacy problem in government agencies: an exploratory study . Proceedings of the 16th Annual International Conference on Digital Government Research , 150 - 159 , ACM.

4. Assia Alexandrova . 2012 . Business requirements analysis and development for legacy system replacement projects in government organizations .

5. M. P. Gupta , and Deepak Bhatia . 2005 . Reworking with a legacy financial accounting system . Lessons from pharma company. Interfaces , 30 ( 3 ).

6. Harry Sneed . 1995 . Planning the Reengineering of Legacy Systems . IEEE Software, January .

7. Steve Frosdick . 1997 . The Techniques of Risk Analysis are Insufficient in Themselves . Disaster Prevention and Management 6 ( 3 ): 165 - 177 .

8. Valerie A . Brown , John Alfred Harris, Jacqueline Y. Russell (Eds.). 2010 . Tackling Wicked Problems, through the Transdisciplinary Imagination . Earthscan Publications Ltd, London.

9. Horst W. J. Rittel and Melvin M. Webber . 1973 . Dilemmas in a General Theory of Planning . Policy Sciences 4 , no. 1969 ( 1973 ): 155 - 169 .

10. Rosalie Ocker . 2010 . Promoting Group Creativity in Upstream Requirements Engineering. Human Technology 6 , no. May ( 2010 ): 55 - 70 .

11. Colin Potts , Kenji Takahashi and Annie I. Anton . 1994 . Inquiry-Based Requirements Analysis . IEEE Software, March 1994 , 21 - 32 .

12. https://www.atlassian.com/software/jira?

14. Umar Ruhi . 2015 . Level Up Your Strategy: Towards a Descriptive Framework for Meaningful Enterprise Gamification . Technology Innovation Management Review , 5 ( 8 ), 5 - 16 .

15. David Osborne . 1993 . Reinventing Government . Public Productivity & Management Review 16 , no. 4 : 349 - 56 .

16. Stephen A. Rains . 2005 . Leveling the organizational playing field-virtually: A meta-analysis of experimental research assessing the impact of group support system use on member influence behaviors . Communication Research , 32 , 193 - 234 .

17. Marta Rauch . 2013 . Best practices for using enterprise gamification to engage employees and customers . In Human-Computer Interaction. Applications and Services , pp. 276 - 283 , Springer, 2013 .

18. Marc McDonald , Robert Musson and Ross Smith . 2007 . The Practical Guide to Defect Prevention . Microsoft Press.

19. Ian Bogost . 2008 . The rhetoric of video games . In The Ecology of Games: Connecting Youth , Games and Learning, K. Salen, Ed., The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning . The MIT Press, Cambridge, MA.