The CURATE! Game Its Development, Evaluation and Use Karolina Badzmierowska Vicky Garnett Susan Schreibman Trinity Long Room Hub Trinity Long Room Hub School of English Trinity College Dublin Trinity College Dublin, Trinity College Dublin Dublin, Ireland Dublin, Ireland Dublin, Ireland badzmiek@tcd.ie vicky.garnett@tcd.ie susan.schreibman@tcd.ie Abstract - The DigCurV CURATE! Game was developed by executives of cultural heritage institutions. The idea for the Katie McCadden, Prof. Susan Schreibman, and Dr. Jennifer CURATE! game was devised following two influences. The Edmond at Trinity College Dublin (TCD), in conjunction with first was the focus groups that were carried out as part of the Carol Usher and Kate Fernie at MDR Partners in the UK. required work for DigCurV, where it became apparent that a Developed as a means to highlight the importance of training in ‘hypothetical scenario’ worked best in eliciting responses from digital curation among practitioners and managers working in the participants on their experiences of digital curation, libraries, museums and cultural heritage institutes, the game has since expanded into a self-assessment tool, a team-building typically within their institutions. The second came from a exercise and a training tool for early career students. A recent poster that was presented by Dr. Jennifer Edmond of the survey conducted by TCD and MDR Partners on behalf of CENDARI project, Dr. Owen Conlan of the cultura project, DigCurV on the use and perceptions of the game has revealed and Katie McCadden who was working on the DigCurV new scope for further work project. The poster, entitled ‘Digital Cultural Heritage and Social Participation’ was presented at the Intel European Keywords - digital curation; digital preservation; serious games; Research and Innovation Conference (ERIC) in 2011 (see training; self-evaluation; role-play; experiential learning; game- Figure 1). based training I. GAMEPLAY AS A TRAINING TOOL Game or role-play has long been established in practical training situations - typically in high-risk situations (Ericsson, 2006). Aeroplane pilots are trained in simulators before they enter the cockpit; military training involves a significant amount of combat training before soldiers are shipped off to the battlefield (Smith 2006). Yet, experiential teaching practices, or 'Serious Games', as a training or coaching technique in less stressful/ high-risk environments have been steadily increasing in popularity (Ritterford, Cody and Vorderer, 2009), be they for training in marketing, strategy development or rehabilitation. Team-building workshops and exercises have also been proven effective, particularly in an online environment (Hirsch 2001; Pantazis, 2002; Grzeda, Haq and LeBrasseur, 2008). II. THE EARLY GAME The objective of the Digital Curation Vocational Education Europe project (DigCurV) is to provide a Curriculum Framework for training in digital curation. The CURATE! game began as an unplanned output of the project, which had already defined its objectives and deliverables at the beginning of the project. The key deliverable, the Curriculum Figure 1:. "Digital cultural heritage and social participation" poster, as Framework, was designed to be used by students and early- presented by Dr. Owen Conlan, Dr. Jennifer Edmond, and Katie McCadden at stage researchers, practitioners in the field, and managers and Intel Eric 2011 The intention behind the poster was to bring these three takes place, and the inner board, which indicates the players’ cultural heritage projects (CENDARI, cultura and DigCurV) progress in the 'Digital Curation LifeCycle'. based within Trinity College Dublin together to present an The outer board is divided into an equal number of squares overview of the ways in which the intersection between on each side, each of which has a scenario or instruction that technology and cultural heritage could contribute to social directs the player as to their next move. In some cases, they cohesion (J. Edmond, personal communication, May 2013). are instructed to select a card from one of three categories; Taking the idea of a game to present a scenario to people in a “CAUTION!”, “DANGER!” or “DigCurV”. The centre of way that would get them to discuss such issues was then the board features a second element of play, the lifecycle of applied to the problem of enabling those engaged in Digital the Digital Curation Project. This is represented as a circle Curation to actively discuss their experiences of working with that is divided into three key sections, "1. Develop, 2. Educate, digital objects. The early version of the game was entitled 3. Manage“. Players move their ‘token’ pieces around this ‘Game of the Digital Curation Lifecycle’. Following its circle as they complete one full cycle of the main game board. inception, Katie McCadden and Susan Schreibman at Trinity College Dublin (TCD) developed the game in conjunction The game was tested with colleagues and feedback was with DigCurV colleagues Carol Usher and Kate Fernie at gathered. The game was then produced as an online download MDR Partners in the UK. The game was developed to include from the DigCurV website34 once the feedback had been taken a suite of questions relating specifically to obstacles or into consideration. achievements typically found within digital curation projects. Answer sheets were also developed and included to allow IV. WHERE IS THE GAME BEING PLAYED, AND WHO IS players to keep a record of their answers. These answer sheets PLAYING IT? were then either collected by DigCurV and used to provide an insight into the way in which digital curators deal with certain Two years on from the development of the game, a survey situations, or they remained with the players for them to use as was carried out by TCD and MDR Partners on the uses of the a reference tool. The title of the game was changed to game, and the reception the game has received. This survey ‘CURATE!’ was designed for completion online, and was made available from April 2013. The survey has, by and large, reinforced anecdotal comments made to the game developers. The results so far indicate that the game has been played across Europe, as we already knew from conversations with players, as well as in the USA and Australia. In Europe, the game has been played in Belgium, Germany, Ireland, Italy, The Netherlands, Spain and the UK. In the United States, we found an instance of the game being played in New Hampshire, and in Australia we received feedback to the survey from Canberra (see Figure 3). The game received exceptionally good feedback online, especially among Twitter users. Figure 2. The CURATE! gameboard III. DEVELOPING AND TESTING THE GAME The game was envisaged as a board game, much in the Figure 3. CURATE! game play around the world tradition of non-electronic games, such as Monopoly and the “Game of Life”. The game board was designed with two key play 'areas'; the outer board on which the majority of play 34 http://www.digcur-education.org To date the game has been mentioned in more than 50 tweets Management and Preservation course.40 Trinity College from all over the world.35 Dublin has included a session of the game as part of its delivery of the MPhil. in Digital Humanities and Culture. Conferences DigCurV partners at the University of Göttingen also included The game is played most frequently at conferences, usually the game as part of their ‘nestor/DigCurV School 2012’ in during a coffee break or poster session. In many cases the October 2012. game is introduced to the conference participants by a CURATE! as a Team Building exercise DigCurV partner who either co-organises the conference, or who brought the game as a part of their poster presentation. Playing the game is also very popular in team-building For example, the game was played at the Digital Strategies for exercises, the third most popular use of the game. For Heritage Conference (DISH 2011) in Rotterdam during the example, a pre-Christmas party for students studying Digital “Digital Curation Training: Mind The Gap!” workshop which Humanities, as well as those interested in digital curation was was co-organised by DigCurV partners Kate Fernie and Katie held in December 2012 at TCD. The game was also played at McCadden.36 Trinity College Dublin demonstrated the game a data management meeting in Utrecht in 2012.41 Further at the Digital Repository of Ireland's 'Realising the examples of team-building include Dartmouth College Opportunities of Digital Humanities' conference37 in Preservation Service in New Hampshire42, who sat down to a November 2012 in Dublin, and at the COIMBRA group game during a coffee-break. In a similar setting the game was workshop 'Digitising University Collections' at the University played by the staff members of the Academic Commons of Edinburgh in May 2013. The game was played by online repository at the Columbia University, New York.43 attendees at the DigCurV ‘Framing the Digital Curation Improvised use of the game Curriculum’ Workshop in Florence in December 2012 and consequently at the final DigCurV conference in May 2013. We have also received responses that indicate that users The game was also played at LATINA Post-it in Vilnius, are improvising in their use of the game. One lecturer decided Lithuania at the end of May 2013, which comprised a seminar to dispense with the game board altogether, and simply used and workshops organised by the Mykolas Romeris University, the game-cards as prompts for discussion and for testing Lithuania and the Oslo and Akershus University College, student knowledge. Norway.38 V. RESULTS OF THE SURVEY The game has also been included in conference In evaluating the game, respondents to the survey were programmes through more organic means, where an individual asked to rate elements of the game, for example the questions, who is not a DigCurV partner has played the game elsewhere, the board layout, and its complexity, while giving reasons for and has included it in their conference. The most recent organic example of including the game in a seminar/training their answers. course is the Ina EXPERT event "FRAME 2013: Future for Interaction/Discussion/Education Restoration of Audiovisual Memory in Europe" which will be dedicated to 12 European professionals in the media industry The discursive and interactive elements of the game were and will take place in Paris in June 2013 39. This is, of course, considered strong for the most part. Criticism was mostly the most encouraging use of the game. geared towards occasions when the game was played with players who possessed very different levels of knowledge in CURATE! in the classroom digital curation. The mixed experience of the players on these Teaching is the second most popular situation in which the occasions might indicate that different levels of knowledge might be barriers for some players to engage in discussion. game is played. Respondents to the survey indicated that they Knowing this, however, might be turned into a strength rather have used the game as part of a university programme. Students of DigCurV partner HATII (Humanities, Advanced than be viewed as a barrier. One of the purposes of the game Technology and Information Institute), University of Glasgow is to educate. By creating a situation in which players with played the game in their final week of the MSc. Information less experience can discuss digital curation issues with their more experienced colleagues, they are given the opportunity to learn and develop their own knowledge base. 35 http://storify.com/karolinabadz/digcurvcurategame 36 40 http://digitaalduurzaam.blogspot.com.au/2011/12/playing-digital-lifecycle- http://blogs.arts.gla.ac.uk/hatii/msc-imp-mcpdm-best-blog-awards-results- game-dish2011.html are-coming/ 37 41 http://www.dri.ie/realising-opportunities-digital-humanities https://twitter.com/leandervdspek/status/185475787611320323 38 42 http://akkordio.net/postit/?page_id=7678 http://dartmouthpreservation.blogspot.ie/2012/07/winning-game-of-digital- 39 http://www3.ebu.ch/files/live/sites/ebu/files/events/Academy/2013/DOCS/1 curation.html 43 .%20FRAME%20programme%20Draft%202013%20session%201%2026022 https://twitter.com/ResearchAtCU/status/193406095505625088/photo/1 013.pdf https://twitter.com/ResearchAtCU/status/193405223610499074/photo/1 Respondents to the survey recognised this: "Some questions are not very inspiring and formulated too broad." "depends very much on the people in your team. You can talk about the issues on different levels." Those with a basic knowledge, the respondents claimed, tended to give very general and broad answers to some of the "The more experienced the players are, the more more complex questions, whereas those with a higher level of interesting is the outcome." expertise in the field found many of the questions to be too Equally, players at a reasonably similar level of expertise broad in scope in any case, and therefore not challenging. in the field as collaborators in a digital curation project will Some felt that the topics of the questions did not cover the have a different background and experience, therefore the entire digital curation experience sufficiently. One criticism game play and resulting discussions are a learning curve for came from someone who believed that the questions focused all. “too much on the side of costs, funding, etc. Not enough Many respondents to the survey felt that the game’s 'fun discussion of risks that would result from strategy selection, factor' creates a friendly and creative environment where etc”. The survey specifically asked if players felt other topic players can openly discuss the topics and self-evaluate. areas should be included, and the majority of results indicated Similarly, the game creates a ‘safe’ environment in which a need for more questions relating to project planning for the players can put forward suggestions on hypothetical situations project, issues surrounding the act of digitalization, without feeling like they might make a mistake. There will establishing and maintaining standards in digital curation and inevitably be some in the group who disagree with an answer a knowledge of software. Interestingly, the same number of given, but this only leads to discussion, as indicated by these people also indicated they felt there should be more questions respondents: on funding (see Table 1). However, the most common issue raised was the small amount of cards provided with the game. "The interactive nature of the game - the questions lead This had an impact on the game overall, as in many cases the to more and more discussion as the game goes on." lack of cards / questions caused many to abort the game before "I think the idea of raising awareness by a game is completing it, as indicated by these comments: great. "Could not be finished. Not enough questions." "A nice way to bring up discussions about "Before finishing the first round there were no digitalization issues." questions left." The Game Questions The questions and scenarios on the DigCurV and CAUTION cards range from positive points (DigCurV cards) for discussion, e.g. “You receive funding for a Digital Preservation project; what’s the first thing you do?”, to quite serious problems (CAUTION! cards) that can potentially arise during the lifecycle of a project, e.g. “There has been a technical failure. Your metadata from the first 3 months of the project is gone and it was not backed up. How will you handle this?” The questions also represent different levels of complexity of problems and issues that can arise. The issue of expertise among players, however, was once again seen as a weakness by respondents when it comes to the questions in the game. Some respondents felt that the quality of answer given by a less experienced player would hamper the learning capacity within a game session of players with mixed experience. " The questions are formulated in such a way that players with little knowledge can give very general answers and get away with it." "Some questions are too general and can provoke very general and superficial answers. Especially when Table 1 -Survey responses "Which topics/subjects would you like to feature played with players with little knowledge about more or less digitisation projects." We have already seen how one teacher dispensed with the board altogether and focused on simply using the cards. This Structure and Design of the game indicates that the link between the board and the questions is The game is designed in the familiar format of a perhaps tenuous in its current form. It also suggests different gameboard, on which boxes or ‘spaces’ are organized in a directions in which the game could be developed. The linear structure for players to move around the board on the imagery and graphics of the board have been designed to throw of a dice. This is a common format for board games match the DigCurV branding of dark blue, orange and white. and makes game play instant and accessible to players. The However, there were some criticisms regarding the size of the three ‘Digital Curation Lifecycle’ stages in the centre were not text on either the gameboard or the questions cards, which considered clear enough for players, who could not understand some felt was too small. their purpose: Access to the Game "It is absolutely not clear what the function is of these stages The main strength for the game in terms of access is the and how they relate to the questions." ability to download it directly from the DigCurV website. A suggestion was made as to how to better make use of However, there were still some problems encountered with the these stages: download, such as this respondent whose colleague spent too much time on printing: "Give different questions for different stages." “Printing all the game cards took quite some time, my The game-flow received a mixed response. The majority colleague told me.” considered the board structure to be ‘ok’, with further equal responses indicating that the structure was either ‘easy’ or VI. FURTHER DEVELOPMENT FOR THE CURATE! GAME ‘difficult’ (see Table 2). The deliberate ‘stumbling blocks’ on the ‘DANGER!’ cards was seen by some as a positive aspect, Taking the feedback from the survey into consideration, whereas it frustrated others: and coupling this with anecdotal feedback we have received through our own game sessions, DigCurV is currently "No 'lose next turn' card, or other things that slows down the developing a plan for improvements to the game that should game." address many of the issues highlighted. Many of the "We decided to sometimes ignore 'lose next turn' because it improvements are possible within the short term for the game, slows down the playing rhythm..." and are considered possible before the end of the DigCurV project. However, some others require much more extensive One respondent commented that they strongly disliked development, and have been proposed as a major part of a new “being stuck in the same place too often (on the board)”. funding application. Short term developments • The quantity, quality and range of the cards within the game was the first issue addressed, and was completed with 41 new cards trialled during a game session at the DigCurV Final Conference (May 2013). • Clearer instructions will be developed in various media, including a YouTube instruction video. • To tackle the issue of the different levels of experience of players, it is proposed that two sets of cards be developed to allow for a ‘genius’ version of the game Table 2 DigCurV Survey Responses "Rate the board" for those with more experience or those who want a The relationship between the questions and the game-board challenge, and a ‘standard’ game for those who are was brought into question by one or two respondents, one of starting out, or for mixed ability groups. Of course, we whom described the journey around the board as ‘kind of would not wish to be prescriptive as to who should play mechanic”. They also took issue with what they described as which set in particular, and it should also be possible a ‘luck’ component that meant that the quality of answers for players to combine the two sets of cards if they feel given to the questions had no bearing on a player’s progress it would make the game more interesting. around the board. The size or format of the game-board itself The increased number of play cards is also hoped to amend proved difficult for those playing in groups larger than 5: the issue that some felt the progress of the game was “The idea of having a game for digital curation is great. hampered by too many ‘lose a turn’ or ‘go back one space’ However, the way it is designed now (a board game) it is a bit cards. However, that said, there was further anecdotal difficult to play it in a group bigger than 5-6.” feedback that indicated that the element of risk, or obstacles to the game made it closer to the ‘real-world’ scenarios and experience of the players. Both the strengths and weaknesses frustrations associated with a digital curation or preservation revealed will be taken into consideration for the continued project. Nonetheless, the ‘DANGER!’ category received the development of the game. While many changes can be made fewest new cards. in the short-term before the end of the project, the main changes to ensure greater flexibility would require a greater Continued additions to the cards are feasible before the end investment of time and money. of the project, and it is hoped that further development will be able to link the topics on the cards to the three ‘lifecycle’ ACKNOWLEDGEMENTS stages on the gameboard. The authors wish to acknowledge the inspiration of Katie Medium-term developments McCadden, Dr. Owen Conlan and Dr. Jennifer Edmond, who Further developments are desired, but may require first developed a game-play poster which in turn, fostered the additional funding in order to complete. For example, re- developed the CURATE! game; and in particular Katie formatting the layout of the gameboard to include more McCadden who took the lead in game development in squares would be feasible within a 6-month timeframe, but are conjunction with Susan Schreibman, Carol Usher and Kate perhaps not possible within the remaining month of the Fernie of DigCurV. project. REFERENCES Long-term developments [1] Ericsson, K. A. (2006). The influence of experience and deliberate Ultimately, it is the aim of DigCurV to produce an online practice on the development of superior expert performance. In K. digital version of the game. Much of the feedback indicated A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), that this would be favourable and this would certainly increase The Cambridge handbook of expertise and expert performance (pp. 685-705). Cambridge, UK: Cambridge University Press access and improve the interactive components of the game. [2] Grzeda, M., Haq, R. & LeBrasseur, R. (2008). Team Building in an Options could include the ability for players to customize the Online Organizational Behavior Course. Journal of Education for game to their needs, developing a version that could be used Business, 83 (5), 275-282. for assessment purposes in training, or creating some manner [3] Hirsch, D (2001). Prepare for the global e-campus. OECD Observer, 229, 57–58. of multi-player set up that could allow colleagues at different [4] Pantazis, C. (2002). Maximizing e-learning to train the 21st institutions to play a game. The online version could play a century workforce. Public Personnel Management, 31(1). 21–26. role of an open forum where the participants can ask [5] Ritterfeld, U., Cody, M., & Vorderer, P. (2009). Introduction. In U. questions, discuss issues, network and learn from each other Ritterfeld, M. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 3-9). New York, NY: Routledge outside of their regular working environment and comfort [6] Smith, R. (2006). Technology disruption in the simulation zone. This could also help to populate the card questions industry. Journal of Defense Modeling and Simulation: database based on the players’ own experience, expertise and Applications, Methodology, Technology, 3, 3-10. concerns around digital curation and preservation. The game has the potential to be widely included in a number of training programmes across relevant institutions and training providers, e.g. higher education courses, digital curation and preservation courses, libraries, repositories, museums, archives, galleries. In order to proceed with this long term development a carefully approached outreach plan, promotion and development strategies need to be created and disseminated. Unfortunately, this is beyond the scope of the current project. In order to achieve this particular goal would require extensive development over a significant period of time. VII. OVERVIEW The CURATE! game has been extremely well received since its development two years ago. So far it has been used mostly in the classroom and at conferences, which has introduced the game to many people. The results of the survey have revealed several strengths of the game that were perhaps unexpected (flexibility of use in training environments by using the cards only), and revealed weaknesses of the game that can be addressed through recognizing the variety of