The game \Enigma Galdiano" is a mobile game that implements a treasure hunt in the Lazaro Galdiano museum in Madrid (Spain). The main design challenge in this game is to promote and facilitate the collaboration between an adult and a kid playing the game together. The collaboration is mediated through a map, a real one, that is used by the adult and contains additional information that is not included in the app. In this paper we describe the game along with the results of an empirical evaluation that suggest a few lines of improvement.
The current challenge for museums is how to successfully turn their institutional
knowledge and authority into meaningful, engaging experiences by leveraging
the appropriate technological media in the context of their physical settings,
and for heterogeneous audiences [
People typically visit museums and heritage sites in social groups, either in
conjunction with organized parties or with family or friends. Mobile devices,
on the other hand, are often understood as antithetical to social interaction,
privileging the personalized experience and hindering engagement between
visitors. An initial e ort towards mobile-mediated social interaction is described in
[
In Enigma Galdiano we propose a di erent approach to promote mobilemediated social interaction. In our game, designed for two participants, one of the participants use a mobile device while the other has a physical map with additional clues for the treasure hunt. The game is designed to make the participant with the mobile device to lead the experience, and have the one with the map as counselor or assistant. The one with the device is supposed to be a kid and the one with the map an adult.
If mobile-mediated collaboration is hard to achieve, it becomes even harder
when in an asymmetrical setting such as the one we describe here: adult and
kid, usually parent and child. Having adults and kids enjoying the visit together
poses additional challenges as described in [
Given the complexities of the game settings, once we developed Enigma
Galdiano, we needed to test our design and verify whether it served the purpose we
intended of letting the kid lead the experience. From the di erent approaches
found in the literature for empirical evaluations of interactive systems in
cultural heritage [
The rest of the paper runs as follows. Next Section describes Enigma Galdiano. Section 3 describes the experimental setup while Section 4 discusses its main qualitative results. Finally Section 5 presents future work and concludes the paper. 2
Enigma Galdiano is a game designed to be played at the Lazaro Galdiano Museum in Madrid. The Lazaro Galdiano Museum is built around the private collection of Mr. Lazaro Galdiano, one of wealthiest persons in the beginnings of the Twentieth century in Spain. The collection consists of a large number of paintings, weapons, coins, and other precious objects from di erent periods and geographical origins, including paintings from El Bosco, El Greco and Goya.
The game is designed to be played in collaboration by an adult and a kid. The kid will be using a mobile device while the adult will have a map with additional information to play the game. The actual game is in the device, and although it is feasible to play the game just with the information provided by the device, it would take much longer. Furthermore, our intention is to provide to the adult with a resource to promote the dialog with the kid about the museum and its contents.
The game is initiated at the entrance of the museum where the two main characters, captain Ann Jack and his partner Sr. Mendoza, a talkative parrot, will welcome the pirate apprentice and ask for her help to nd the treasures in the Galdiano islands. Figure 1 shows the two rst screens of the game. The main game mechanic in Enigma is that of searching for certain objects in the museum, given some hints about what has to be found. Target objects are recognized by the mobile device camera, using image recognition software.
Just after the rst object is recognized by the game, which is a small poster about the game itself at the museum entrance, the game advices to the user to go and get a map that will make things easier. Figure 2 shows the map. It is the map of the Galdiano islands, that consists of 4 islands, one for every oor in the Galdiano museum. In every island there is an element for every object that the game will ask to nd, with a visual hint about what needs to be found and the number of the room where this object is placed. Every room in the museum has a number which is showed in a big sign along with information about the objects shown in that room.
After recognizing a given object in the museum two things may happen in the game. Either the character in the object, usually a painting, provides a new hint to advance in the treasure hunt, or a mini-game has to be played in order to advance in the game. Mini-games provide an additional entertaining component and provide additional mechanics beyond the nd-and-point with the mobile device.
Figure 3 shows two mini-games. To the left there is the \hunt the elf" minigame where an elf is moving across the real time image of the painting seen through the camera of the device. The player has to tap with her nger to hunt the elf. In some cases the elf may jump to another painting and the player will have to nd it and repeat the process. In the right side of Figure 3 an example of the \spot the di erences" mini-game is shown. In this case, the mobile device shows a modi ed version of the painting just recognized by the user, what implies that she is in front of the painting and can spot the di erences.
Figure 4 shows two additional mini-games. To the left an image of a desktop from the Eighteenth century is shown; when the player is in front of the actual desktop in the museum, she has to tap on the drawers of the desktop in the right order. In this case the map is very useful because it shows numbers with the answer. To the right in Figure 4 a multiple-choice question is shown. The answer to the question is found in the museum card by the object that has just been recognized. Multiple-choice questions such as this has a very important role in the game because they force the kid to interact with the elements of the museum and read the museum card. This way we can put some factual knowledge into the game, teach the kid about sources of information in a museum and slow the pace of the game since many kids tend to run from place to place, not even noticing they are in a museum.
Figure 5 shows two \magic rooms" in the game. A magic room in Enigma is a 3D scene where the camera is set in the center of the room and the player may look around in the 3D scene by moving the mobile device, what makes the camera to move in the same direction. In case the mobile device does not have a gyroscope, the game detects it and shows a virtual joystick in the screen, such as the ones shown in Figure 5, that allows to control the camera. Magic rooms connect the real world in the museum with the characters and script in the game, in order to promote player immersion.
Enigma combines the core mechanic of object recognition in the museum with the mini-games we just described, for a total of 16 objects to be found and 13 mini-games in between. The game takes approximately one hour to play and gives the visitor the chance to admire a small selected fraction of the museum contents. From a pedagogical point of view the main goal of the game is to make the kid comfortable in and curious about the museum, or as stated by one of our small visitors, an 8 years old kid: \it transforms a boring museum into a cool place".
Once we have got to make the kid comfortable in the museum, we o er the adult the possibility to extend the game by getting additional \achievements", and keep on exploring the museum at a a slower pace. Typically, game achievements are optional additional goals that the most motivated players want to achieve in order to distinguish themselves from those other players that just get the compulsory goals of the game. Figure 6 shows additional achievements in Enigma, a number of additional works in the museum that can be recognized by the game, and once discovered, a short anecdote related to the work history or its contents is unlocked in the game. For example, in a British painting from the Eighteenth century depicting a noble lady, the lady has an orange in her hand, because at that time oranges were so expensive in England that having one was a clear sign of distinction. 3
The experiment was settled to control the path of the visitors and their actions while they play. For that experiment the unit was conformed by an adult and a child. It began with a welcome to the museum and a quick informed consent, afterwards, they were provided with a tablet with the game installed in it. They were informed that it was not crucial to nalize the game for the experiment. It was established that 1 hour was the maximum time to use the device and walk through the museum. We were aiming to observe:
2. The use of the map as a tool. 3. The use of the device as a tool. 4. The time invested in each quest. 5. Technical problems. 6. Followed paths by the players.
The sample was formed by 8 pairs of adult with child. Age of the children were between 6 and 12, more speci c there was a 6 years old, two 7 years old, two 9 years old, a 10 years old, and two 12 years old. The sample was composed by 6 boys and 2 girls. The age of tutors was not considered relevant for the experiment but they were between 33 and 51, gender balanced between mothers and fathers.
We used three di erent experimental instruments: observations of what the players were doing, questionnaires at the end of the visit, and analytics obtained from logs in the app. 3.1
Observations Each experimental unit was followed by an investigator. It was necessary to build a map that contains actual information about the exact place where the art work was at the moment in the museum. The investigator was provided with an up to date map of the museum with all the exhibited works, it was her duty to register which ones were looked, which ones were just passed by and which ones were just ignored. It was also her duty to register the times that the couple used the device, talked about the device, pointed through the device to wrong artworks as well as the number of times they used the map as a tool to solve problems, the number of times they talked about a tool, or the number of times they used the game help button. All that information was registered in the map at the museum location where it happened, that way we can consider when the unit was checking their tools and the available resources.
Researchers were trained about how to use the map and the legend before they started the experiment. There was three observers. The experiment was designed to be run simultaneously as many times as observers were able to be in the museum with the experimental unit (the tutor and the child). The most common was two at the same time but at some point there were three researchers at the same time.
The fact that several experimental units were running the experiment at the same time had an interesting and unexpected consequence on the experiment. We talk about it as the \racing e ect", and it is the competitively atmosphere that was generated between groups. It is convenient to think about the possibility of a ranking or other kind of gami cation for future work. In those occasions the racing e ect was registered as a \Disruption" and the verbal clues between teams as \Help". That was the rst part of the observation, what is complemented by the two following ones. It is the most qualitative part, it was supposed to cover the performance actuation. Table 1 shows a summary of the observations, using this legend: { MH. Map help: Looking for help at the map. { DP. Device Pointing: Pointing with the tablet to an art work. { DH. Device Help: Looking into the device for help. { H. Help: Getting help out of the device or the map. { D. Disruption: Any disruption from the main task. { AP. Adult participation: The adult is holding the device.
7 8 7 16 14 30 8 9 6 1 4 7 0 3 Once the playing time was over, we had two questionnaires for the evaluation, using a 10 points Likert scale. Questionnaire number 1 was for adults to respond and it was intended to measure three concepts: satisfaction with the experience, usability of the game and interaction between the adult and the child. In addition there was a second part in the questionnaire with open questions. That part was designed to collect the adult opinion about not only the game but also how to improve it or whom can it be recommended to.
Questionnaire number 2 was for children, designed for the children to answer on their own, evaluating their favorite parts of the game, including not only simple questions but also asking them to draw a part of the game of their choice. Results from questionnaire 2 were all between 8 and 10, demonstrating the satisfaction of the kids. A summary of the average responses to questionnaire number 1 is shown in Table 2 #
Question 1 I've enjoyed the museum thanks to the game 8.75 2 My child has enjoyed the game 8.75 3 The parrot has been helpful 8 4 The map has been helpful 8.625 5 I've been able to see other works not included in the game 6.25 6 My child required help from the map 6.875 7 My child not always understood the clues 5.375 8 The game was di cult for my child 5.125 9 The idea of the application is good 8.625 10 I like the idea of using a game for visiting the museum 9.25 11 I like the script of the game 8.5 12 I like the characters 8.375 13 Visiting the museum with the game is fun 9.125 14 The length of the game is adequate 8.25 15 Clues are helpful 7.75 16 I understood the islands metaphor 8.875 17 I understood the relation of the map with the works 7.875 18 I liked the mini-games 8.875
In general the game was well rated by the visitors, it was general the feeling that the visit improves with the game, as we can see in questions 1, 10, 14, 17 and 18. The part that was rated as the best one were the mini-games. There was a third method of compilation going on during the experiment, analytic information was saved through the experience. It was remotely saved by the Google analytics plug-in installed in the game. We mainly measured the time employed for every quest in the game, with the results shown in Figure 7.
These numbers are a useful tool to evaluate the di cult points of the game. As it is shown in Figure 7, there are some di erences between quests. Those di erences are not always real, quest 7 is a compilation of three quests that are run at the same time. Quest 10 is also a compilation of 2 quests in the game. Having those points under consideration we can see that all quests but quest 4 and quest 6 are completed in between 1 and 5 minutes.
Quest 4 is located the furthest as possible from quest 3, quest 3 is at the ground oor and quest 4 is at the third oor. Quest 6 is the one where the map gets really relevant. The problem can be that, when they arrive to that quest, their dynamic is already settle and it is di cult to change. That is the reason why their timing gets very high, because that quest is designed to be very hard without the map. The map was design as a tool for the interaction, our proposal was to give adults useful clues for them to support children. It was supposed to be an interaction focus. The experience shows another reality, the team interaction with the map can be understood as a sign of the team character. One of the reasons why the map does not show out as a useful resource can be the way that it was designed. We were aiming to introduce players in a full treasure hunt experience, with a non so obvious treasure map. The reality shows that the interaction between the app and the map is not intuitive so it needs to be trained. For future work it can be interesting to improve the map with an easy legend in order to force a bit more the interaction device-map.
The team character is re ected in the way they use the map during the experiment, in our opinion there are two types of teams: the re exives and the experimental ones.
The re exive team is the team that has a full understanding of the tools that they have been provided and they use all of them. There is only one example of that kind of player in our experiment, but their results are clear enough to show the character of the team. It is the couple number 3 in Table 1. Their use of the map is active, as we can see in they consulted it 25 times. That was the reason why they only used the tablet 12 wrong times. It was also the reason why they used the device help more than any one else, 10 times, because they were pointing with the tablet only to the most probable art work. Searching for the answers to the given clues. That was also re ected in their path in the map, their movements were straight to the point. That was the designed behavior but unfortunately was the less common.
The experimental team has an out of control device pointing behavior. They visit the museum pointing at every single art work that can be answering what they are looking for. Not paying much attention to the map or the device help but pointing with the tablet up to 32 times wrongly. In the map path it is easy to see random movements, coming and going, and a higher use of external help (museum workers and museum signs).
It is interesting that despite the number of times that they have consulted it, or the help that it has provided them, ve out of eight people prefer the actual physical map than an integrated virtual map in the app. The average punctuation about the satisfaction with the map and the help from it is 8.7. It shows that the perception of the users is not always related to the real help that they found in the map.
About the team roles there are some interesting evolution on them as a result of the device use training. Leading the experience is not always into the adults hands, we found two stages: { In the rst stage, the child begins to understand certain codes and the game ow, but only makes suggestions on what step to take next, while the adult still continues to maintain control of the activity, reading app and map at the same time. { In the second stage, the child already understands the codes and the game dynamics, making e ective decisions. As these decisions become more and more successful, the adult begins to take a supporting role with the map, because the child is the one on control with the device. Finally the child becomes the leader of the activity, turning to the adult when the game speci cally requires it.
This development has been observed in multiple tests with the exception of children under 8 years old. None of our under 8 years old children (couples 4, 5 and 7) were able to nish the activity in the given time (1 hour). It might be because none of them were able to reach the second stage of team roles, they were stuck in the rst stage where the parent leads the activity not delegating responsibility on the child. That fact is also shown on the questionnaires, questions 9 and 10, where they are asked about the di culty of the game.
It can be interesting, for future work, to consider a deeper analysis of the interaction between adult and child, and how the adult trusts the capability with the tablet. Only in one case the use of the tablet was, more or less, shared. In most of the cases, the adult only interacts with the device when the anxiety level is high. Otherwise it allows the child to explore and try as much as he or she wants. The adult is more pro-active into asking for external help, going to the museum employees, reading the maps of the museum or the museum cards. 5
We have presented the collaborative game Enigma Galdiano and the main results of an empirical experiment about its use. The main conclusion of this experiment is that players do not always perform the interactions as expected. In the design of the game we wanted to nd a balance between challenge and easy of use. For that reason, the rst quests in the game are designed to introduce the interaction between the app and the map, leading the players to discover the meaning of the numbers in the map: they represent the room number where the artwork is to be found. However, as the rst quests are the easiest in the game, they can be solved without the help of the map, and therefore most of the players do not use the map in those rst quests. From quest 4 on, when things get harder, players have to both understand how the map relates to the app and, their main goal, solve the quest. To make things even more di cult for the game designer, the game was designed in such a way that it can be played without the map, and therefore just one mention to the map is made in the game.
This di culties have already been identi ed in [
Regarding future work, we plan to explore new ways of mobile-mediated collaboration, focusing of groups of kids, where we do not have the problem of asymmetrical relations, and will be able to emphasize the pedagogical content of the game. The authors would like to thank Prof. Mikel Asensio from Autonoma University of Madrid for his help in designing the experiments described in the paper.