RE-PROVO: An Evaluation of Gamification in a Law Enforcement Organization Assia Alexandrova Abstract Computing and Communications Government organizations rely extensively on legacy The Open University systems for their operations. When such systems are Milton Keynes, UK phased out, the new applications which replace them aalexandrova@fortlauderdale.gov often replicate legacy functionality unnecessarily, Lucia Rapanotti resulting in inefficiencies and missed opportunities for Computing and Communications innovation. A prototype of an online discussion game The Open University designed to promote the analysis and critique of Milton Keynes, UK functional requirements for legacy system replacement lucia.rapanotti@open.ac.uk and encourage creativity, was evaluated in a local law enforcement agency. The preliminary findings of the Ivan Horrocks Engineering and Innovation evaluation are discussed, and the potential effects of The Open University gamification on the future of organizational Milton Keynes, UK communications and decision-making are considered. ivan.horrocks@open.ac.uk Keywords Requirements Engineering, Legacy Systems, Gamification, Bureaucracy, Public Sector ACM Classification Keywords Human-centered computing: Social engineering (social sciences); Human factors; Requirements/Specifications Introduction: The Legacy Problem in when business users and IT staff alike are deriving Government Agencies requirements for either commercial off-the-shelf Government agencies are traditionally associated with (COTS) or bespoke systems by drawing directly from bureaucracy, inertia and outdated information legacy systems specifications, usage manuals, or even technology (IT) systems [1]. Legacy technology is legacy code (since often the only place business rampant in public sector organizations. “Green screen” processes and rules are documented is in software code mainframe applications, and non-web based systems [5]). This is detrimental because government are particularly prevalent in law enforcement and public organizations miss the opportunity to revisit, update safety institutions [2]. Many of these systems are in the and streamline their workflows, business processes and process of being modernized, or replaced, and such operational practices, and to be innovative. projects are costly and time-consuming. When agencies undertake technology modernization, the new During the requirements phase in legacy system applications which are being implemented often mimic replacement projects in government agencies, often the the old legacy systems which they are intended to requirements for new systems are not even discussed - replace. This occurs for several reasons: existing instead they are virtually “mot à mot” (word for word) processes are being recreated so that users do not derived from the legacy system’s features. The project have to be retrained, old data schemas are being management philosophy in such cases is to move retained and extended for purposes of compatibility everything to the new technology platform as-is as a with other legacy systems, old features are being first phase, and to consider potential changes and preserved to comply with existing legislation [3]. improvements later, as a second phase [6]. Phase 2, Functional and data specifications are also replicated however, rarely occurs due to budget constraints or because agencies fear that change will lead to changes in IT project priorities. In some cases, operational destabilization, chaos or unintended requirements discussions do in fact take pace, and outcomes both for the internal users of the system, and when they do, proposals for departure from the status- for the public. The phenomenon of risk aversion, and quo are commonly rejected by someone in the uncritical acceptance of an organization’s organization who takes on the role of “devil’s advocate” operational/business process status-quo which leads to and brings up all the potential negative, or catastrophic the de-facto reproduction not only of legacy technology effects of doing things differently from how they have and data models, but also of antiquated organizational always been done. On the other extreme, there are work processes, has been defined as the legacy agency executives who issue directives to make problem of the public sector [4]. innovations - i.e. implement new trendy technology, without regard for the actual impact that this may have The Requirements Phase of Legacy Replacement on operations or on the services provided to the public. Projects The legacy problem manifests itself during the These two diametrically opposed positions - the requirements phase of legacy replacement projects, naysayer, who fears all change, and the reckless innovator who fails to perform an impact analysis – do not uncritically duplicate the legacy system that is correspond to stereotypical attitudes towards risk [7] being replaced. Gamification may encourage and to organizational personas that tend to stifle participants not to “take the path of least resistance” productive functional requirements analysis and and automatically adopt the “safest” approach, but to elicitation sessions in organizations and to introduce suggest changes that take advantage of new emotional conflict which endangers constructive technology, and introduce efficiencies. By anonymizing deliberation. Additionally, project participants’ locus in online deliberations, and introducing incentives for organizational hierarchy tends to be an influence on players to contradict and argue with current whether their concerns are even voiced in such requirement formulations, the game design tries to discussions [3]. downplay, or disable influences that tend to constrain discussions and brainstorming in traditional formats A potential approach towards addressing this issue is to (e.g. peer pressure during in-person meetings)[10]. enable public sector practitioners to be creative during the requirements phase, and to explore innovative A requirements game - RE-PROVO (Esperanto for re- alternatives in depth when discussing and analyzing test), was designed as a gamification layer to the Potts business requirements for applications that are meant et al. requirement inquiry cycle [11]. According to Potts to replace legacy systems. Transdisciplinary and game- and his colleagues when a requirement is initially based approaches have commonly been adapted to defined, it must go through a critique – i.e. a address “wicked problems” [8]. Wicked problems “challenge” to its current contents, followed by defined as issues of a complex techno-social nature [9], subsequent analysis and discussion, and resulting in its exhibit characteristics similar to those of the legacy morphing into a different version. The resulting problem – they are intractable, contradictory and have morphed version can be challenged and reformulated shifting formulations. Gamification can be applied to as well, and the cycle can (should) repeat until an ensure that during the definition and deliberation of improved and agreed-upon version of the requirement requirements affecting work processes in government is arrived at. We borrowed from the inquiry cycle, and organizations, arguments for the innovation of existing used its “challenge” construct as a game action. Two workflows and operational procedures are expressed user/player roles were also established to structure the and considered when specifications for new systems are discussion specifically along the themes of change and developed. status-quo preservation - innovators and heritage keepers. The players in the game are randomly RE-PROVO - Gamifying Inquiry-Based assigned to one of these roles, and two teams are Requirements Analysis formed. The heritage team must issue heritage, or We have hypothesized that introducing game elements legacy-preservation challenges, through which it into requirements discussions - elements such as roles, critiques any one of a set of requirements listed in the teams, points, badges, and anonymizing participation, game application by identifying in them issues that may would result in the development of requirements which lead to risk, operational instability, substantial changes to standard operating procedures, or departures from organization was rather unusual and a departure from existing policies and legislation. The innovations team traditional IT project management practices. The use of must issue innovation challenges, which critique the game dynamics in the discussion and analysis of requirements for being too “faithful” to the status quo, requirements for new technology features and or for replicating inefficient processes, thus not taking applications revealed thought-provoking insights into advantage of new technology to streamline workflows. the impact of gamification on organizational After challenges have been issued, the players should communications and decision making. Three main morph the requirements, so that the critiques raised in themes emerged: 1) employee engagement in group the challenges are addressed. Morphs can also be deliberation and collaborative analysis, 2) the effect of challenged in their own right, and the discussion cycle power relations on creativity and innovation, 3) for them can be repeated. For each action - embeddedness of gamification in core operations. challenging, morphing or commenting, the players receive points. As the points accumulate, the players Employee Engagement in Group Deliberation and can also be awarded different types of badges. After an Collaborative Analysis agreed-on time frame, the players are enabled to vote Participants in the RE-PROVO evaluation more readily on the requirements and morphs so that a winning engaged in online discussions, and they felt that the version for each requirement thread is elicited. All the game elements were interesting to explore. points are tallied by team and by individual, and a winning team and “most valuable player” (MVP) are Employees often have difficulty voicing their opinions announced. regarding how their organization should function - they might feel their suggestions will be disregarded, or they JIRA’s issue tracker [12] (by Atlassian) was customized might be afraid of how others will perceive them. And and extended with a gamification add-on - Jiraffe [13] when deliberating on information technology issues in (by BugPotion), and the resulting functionality was particular, if they are business users of systems - they used to evaluate the game concept in sessions with might feel incompetent or intimidated. Therefore, any practitioners from a law enforcement government additional measures to encourage engagement and agency. While the evaluations of the RE-PROVO discussions and brainstorming would help the prototype are still ongoing, some preliminary findings organization successfully solicit feedback. Introducing have been already singled out at this phase of our game dynamics tends to promote participation initially, research. but with repeated uses of gamification, the novelty of format may subside. Umar Ruhi asserts that enterprise Evaluation in Law Enforcement Information gamification design must be meaningful if it is to Technology Projects sustain involvement and result in prolonged interest RE-PROVO was tested in a local police department. The [14]. An important question becomes whether employment of a game for purposes of IT requirements gamification will need to become permanently elicitation and development in a public safety embedded in organizational processes which require enhanced employee engagement and pro-active online medium made this possible. During the involvement. Will engagement on behalf of the evaluation, our participants did make attempts to guess organization’s workers no longer be assumed to be part or uncover the others’ identities (their screen names of one’s work performance, instead perpetually were fictitious) by exploring various sections of the requiring some sort of incentivization or extrinsic application. This suggests that identity and one’s stimuli? While games and tool gamification may have a position in the organization are important determinants positive effect on individual projects or work processes, when evaluating others’ ideas, comments or critiques. it must be asked whether a hunger for game dynamics In RE-PROVO we wanted the players to assess the in all enterprise workflows or projects is sustainable or requirements, challenges, morphs and comments on desirable. their own merit; we also wanted participants not to be afraid to challenge anyone or argue with others. In Effects of Power Relations on Creativity and Innovation technical discussions in particular, alternative designs RE-PROVO participants felt that anonymity was and architectures can be more easily assessed from a beneficial, but they were nonetheless interested in purely technical perspective, without reference to finding out who the other players were. additional information such as the background of the person making the suggestion. In this sense, Organizations in the public sector are increasingly anonymous online discussion tools with gamified facing pressure to be innovative, to do “more with less” elements that promote competitive behaviors and and to “think outside of the box” [15]. Given the productive conflict, have the potential to subvert legislative constraints and the lack of public trust they traditional open, face-to-face methods that seek to often encounter, this is a sufficiently complex elicit innovations and creative solutions. Will challenge. It has become a regular occurrence for gamification help support participation in organizational employees to be called upon to give ideas, get involved innovation by those who are more introverted, or in in suggestions to overcome problems, propose creative lower positions in the organizational chart? Perhaps solutions, and participate in brainstorming sessions. gamified organizational tools that support group However, as indicated by participants in the RE-PROVO deliberation and decision-making can become the “true game evaluation, there is apprehension to share equalizer” [16]. This is particularly relevant in law opinions and ideas whenever management or agency enforcement agencies where chain of command executives are present. When individuals who are considerations may preempt solution or idea quality. positioned high in the organization’s hierarchy are present in meetings to define systems requirements, Embeddedness of Gamification in Core Operations they (often unintentionally) stifle discussion. Employees The participants in the RE-PROVO requirements game may just echo whatever comments managers make, or evaluation were interested if their winning morphs they may refrain from showing their disagreement. RE- would be actually implemented, or if the game is just a PROVO was made anonymous precisely so that power simulation of a requirements elicitation process. relations do not become a factor in deliberations. The The theme of operational embeddedness of games in Conclusion the enterprise refers to the manner in which gamified A requirements deliberation game - RE-PROVO, was tools and processes result in the creation of a product, prototyped to evaluate if elements such as role-play, or an actionable item. A significant number of games, teams, points and badges can assist practitioners in or game-based applications primarily affect areas that government organizations to tackle the legacy problem, are ancillary to core operations, i.e. they enable and facilitate the analysis of functional requirements for educational activities and training, brainstorming, or the replacement of legacy systems. The evaluation employee networking [17]. There are some examples raised important issues related to the role gamification where games introduce incentives in sales, or customer can play in organizational communications and decision service performance, or are integrated in a quality making in the workplace of the future. Gamified tools assurance process (e.g. employees are encouraged to and work processes have the potential to be fully detect more issues, or software bugs [18]), and these integrated in core production-level processes, and to are indeed the main functions of the company. In our subvert traditional hierarchical decision-making. In law case, the game try-out was undertaken for research enforcement agencies, which rely on strict command purposes, and even though it contained real scenarios and control structures, gamification may promote and requirements from actual ongoing projects, it was improved organizational agility and lead to more primarily an exercise in deliberation, and its outcomes innovative outcomes. have no guarantees of impacting the agency’s IT and law enforcement decision makers. RE-PROVO would be, References in effect, a rehearsal for future discussions, just as 1. Vincent M. F. Homburg. Red Tape and Reforms: many other games or gamified applications are Trajectories of Technological and Managerial primarily educational, training tools. This echoes the Reforms in Public Administration. International notion of “procedural rhetoric” introduced by Ian Journal of Public Administration 31, no. 7 Bogost [19], which posits that the main impact of 2. Hsinchun Chena, Jenny Schroeder, Roslin V. games is to imply and teach a certain procedural model Haucka, Linda Ridgeway, Homa Atabakhsh, Harsh of the world. It would be a relevant line of inquiry to Gupta, Chris Boarman, Kevin Rasmussen, Andy W. determine if gamification can involve more than Clements. 2003. Coplink Connect: Information and knowledge management for law enforcement. “procedural rehearsals” of the organization’s core CACM, 46(1):28–34. processes, but could be directly integrated in decision- 3. Assia Alexandrova, Lucia Rapanotti, Ivan Horrocks. making (e.g. versions of systems requirements with the 2015. The legacy problem in government agencies: most votes in the RE-PROVO game would automatically an exploratory study. Proceedings of the 16th become a part of the new system’s specification Annual International Conference on Digital document). In such a scenario an organization’s Government Research, 150-159, ACM. decision-making processes would be impacted 4. Assia Alexandrova. 2012. Business requirements substantially by game dynamics, and gamified activities analysis and development for legacy system will be, in fact, more than “just a game.” replacement projects in government organizations. In Requirements Engineering Conference (RE), 17. Marta Rauch. 2013. Best practices for using 337-340. enterprise gamification to engage employees and 5. M. P. Gupta, and Deepak Bhatia. 2005. Reworking customers. In Human-Computer Interaction. with a legacy financial accounting system. Lessons Applications and Services, pp. 276–283, Springer, from pharma company. Interfaces, 30 (3). 2013. 6. Harry Sneed. 1995. Planning the Reengineering of 18. Marc McDonald, Robert Musson and Ross Smith. Legacy Systems. IEEE Software, January. 2007. The Practical Guide to Defect Prevention. Microsoft Press. 7. Steve Frosdick. 1997. The Techniques of Risk Analysis are Insufficient in Themselves. Disaster 19. Ian Bogost. 2008. The rhetoric of video games. In Prevention and Management 6(3): 165-177. The Ecology of Games: Connecting Youth, Games and Learning, K. Salen, Ed., The John D. and 8. Valerie A. Brown, John Alfred Harris, Jacqueline Y. Catherine T. MacArthur Foundation Series on Russell (Eds.). 2010. Tackling Wicked Problems, Digital Media and Learning. The MIT Press, through the Transdisciplinary Imagination. Cambridge, MA. Earthscan Publications Ltd, London. 9. Horst W. J. Rittel and Melvin M. Webber. 1973. Dilemmas in a General Theory of Planning. Policy Sciences 4, no. 1969 (1973): 155-169. 10. Rosalie Ocker. 2010. Promoting Group Creativity in Upstream Requirements Engineering. Human Technology 6, no. May (2010): 55-70. 11. Colin Potts, Kenji Takahashi and Annie I. Anton. 1994. Inquiry-Based Requirements Analysis. IEEE Software, March 1994, 21-32. 12. https://www.atlassian.com/software/jira? 13. http://bugpotion.com/ 14. Umar Ruhi. 2015. Level Up Your Strategy: Towards a Descriptive Framework for Meaningful Enterprise Gamification. Technology Innovation Management Review, 5 (8), 5-16. 15. David Osborne. 1993. Reinventing Government. Public Productivity & Management Review 16, no. 4: 349-56. 16. Stephen A. Rains. 2005. Leveling the organizational playing field—virtually: A meta-analysis of experimental research assessing the impact of group support system use on member influence behaviors. Communication Research, 32, 193-234.