Audio Guides And Human Tour Guides: Measuring Children’s Figure 1: One part of the Whale Hunting Exhibition's room (left), Engagement & Learning At A Museum and a mark on the floor indicating an audio local (right). Setting Abstract Vanessa Cesário In this poster we present a study exploring the benefits University of Porto / Madeira Interactive of using audio guides instead of human tour guides to Technologies Institute foster children’s engagement and learning outcomes Madeira Island, Portugal during their visit to a natural science museum. We Figure 2: Children in Group A vanessa.cesario@m-iti.org conducted an experiment in a museum with 25 children wearing numbered stickers and from a school (9-10 years old) in order to discover if using the audio guide device. António Coelho there was any effect of the usage of audio guides University of Porto regarding children’s engagement and learning Porto, Portugal outcomes. We did a pre-test and post-test analysis for acoelho@fe.up.pt learning, applied scales and conducted qualitative observation. We found that children using the audio Valentina Nisi guide were keen to repeat the tour. We also noticed Madeira Interactive Technologies Institute differences in the learning outcomes (the group without Madeira Island, Portugal valentina.nisi@m-iti.org audio guide had better results), although it was not statistically significant. We conclude the poster with a set of suggestions to increase enjoyable learning Copyright is held by the author/owner(s). experiences for children when using audio guides in a CHItaly ’17, September 18-20, 2017, Cagliari, Italy. museum tour. Author Keywords Figure 3: Map of the exhibition museum, learning, engagement, audio guides, human and location on the specific tour guides, children, enjoyment artifacts ACM Classification Keywords enables children to explore, elaborate and expand their MEASUREMENTS B.4.1 Data Communications Devices; K.3.1 Computer knowledge within the museums. The independent variable was to Uses in Education carry out the museum visit with Experiment location or without an audio guide. The Introduction In order to better understand what is the effect of the dependent variables were the Museums are slowly but surely moving away from the usage of audio guides regarding children’s engagement engagement with the exhibition metaphor of being just collections of artifacts to and learning outcomes we conducted a study during and the learning outcomes. become centers where people can engage and the Whale Hunting exhibition (Figure 1) at the Whale empower their knowledge by discovering and Museum from Madeira Island which has audio-guided Engagement: To measure challenging themselves [3,4]; visitors are turning from tours to engage their visitors. The audio guide devices engagement we used validated passive to active participants [5,9]. Usage of audio were built both for children and adults’ usage. The scales from the User Evaluation guides at the museums dates back to the 1950s [11], museum uses two different approaches for the audible Toolkit (the Again-Again Table to allowing museums’ audience to indulge in narratives content, one for adults with an adult narrator voice, measure engagement, and the that aid the construction of meaningful memories as and another for children with young voices in a Smileyometer to measure well as providing the fulfillment of a complete dialogue with each other, prompting the listener to enjoyment). experience. Although the museums’ spatially search for items and to take an accurate look at them constructed narrative might be present in a logical and better and understand several scientific or historical Learning: The learning goal of consistent way, not all visitors choose to follow, learn concepts tied to the exhibit. Near each artifact or set of the tour to the museum was to and engage with it [10]. Though, there are absolute artifacts, there are marks on the floor in order to know about the whale hunting’s cases where the visitor’s mental model discords with prompt children to go to the specific location and listen process through the last years as the design of the exhibition that they are interacting to the podcasts (Figure 1). This is due to the nature of well as the materials and items with [6], changing the focus from experience onto the audio guides being activated by sensors. In the adults’ which could be derived from the guide’s design itself, interrupts the flow [2] which version, the user has the choice to type a specific code whale hunting. We carried out a eventually leads to frustration and poor user that is near the art piece to get more detailed general pre-test/post-test experiences. Therefore, learners might fail to information about that particular piece. On the assessment of their knowledge elaborately appreciate the exhibits due to its contrary, the children’s version remains unresponsive before and after the visit, abundance of precious information and time constraints to any manual inputs. regarding the same issues with which may ultimately lead to information overload [1]; the same level of difficulty. or due to the lack of interest of visitors in the Sample interpretation of the exhibit [8]. Having said that, We focused on 9-10-year-old children taking advantage Qualitative data: In the end of museum oriented learning is a quintessential research of the highest children’s target groups who visits this each tour, the researcher asked topic in the field of informal learning, and those poor particular museum. The sampling method that was to each group at once if they user experiences that could occur during this learning chosen is a non-probability based method, mainly a enjoyed the tour and why in process need to be related, understood and dealt with convenience sample. After obtaining consent from one order to have qualitative in order to craft an informal learning environment that single school we selected 25 children between 9-10 feedback about the experiment. years old of two different classes. The children had never been in the chosen museum before. The users in (M=15.42; SD=28.01) was greater than those who ANALYSIS the same class knew each other; hence to minimize made the visit with the audio guide (M=2.23; This section explains how the this bias, we randomly picked the students and then SD=28.28). This difference was not significant T(N-1)=t, statistical analysis was made categorized them into two groups: one for conducting p>0.05, with a small effect size of 0.24. Regarding this regarding the chosen the visit with the audio guide and a human guide sample, we can infer that a visit without the audio measures. (Group A) and another one to perform the same task guide can increment the learning same as a visit with only without any digital device (Group B). After all the an audio guide does. Although these preliminary results Again-Again Table: it was students have been assigned to a group we went on to go in this direction, further studies with a larger sample applied to check if the children issue them with numbered stickers which they stuck on are needed in order to clarify these results. were keen to repeat the activity their outfits in order to easily identify their group again, reflecting their (Figure 2). Group A contained 13 participants (8 males, Qualitative results: verbal appreciation engagement. In the table users 5 females). Group B contained 12 participants (4 Group A generally enjoyed the tour with the device just needed to select one of the males, 8 females). With Group A, the tour lady telling that it was a fun way to make the museum tour. following options: Yes, Maybe, prompted the users to go to the specific places to hear Nevertheless, some of the children told that the audio No. We computed its the narrated audio piece; with Group B, the lady herself guide had too long podcasts. The general verbal frequencies. narrated specific stories and concepts when near each appreciation was at follows: “I loved it!”; “It’s funny!”; Smileyometer: it was artifact (Figure 3). “It speaks a lot!!”; “At the beginning I thought that the prepared to elicit children’s tool was a cell phone for us to talk to each other…”. opinion/enjoyment on the Results This last comment was due to the physical aspect of overall activity. This was only a Quantitative results: engagement, learning the audio guide device (Figure 2). Regarding Group B, question where children needed On the Again-Again Table [7] the value of the Chi- at the end of the tour, this group was prompt with the to rate on a 5-point Likert scale Square is 8.440. This value was significant (p<0.015). possibility of having had done the visit with an audio how much they enjoyed the This significant result indicates that there was an guide. Hence, they were asked about their preference event. Each smiley was then engagement associated when going on a visit with the for doing the visit with or without the audio guide. They scored as 1=awful; 2=not very audio guide. Therefore, and for this sample, we can enjoyed the visit without audio guides in general, good; 3=good; 4=really good; conclude that the type of visit performed significantly although most of the group would like to try the visit 5=brilliant. influenced the children: they would go to the museum with the audio guide. However, they liked the tour lady Learning: we computed the again for the audio guide but not for a visit without this explaining and the fact that they could talk with their difference between the first tool. Regarding the Smileyometer [7], although we had friends while performing the tour. The general verbal test with the second test to higher values on ranks, the Test Statistic showed us appreciation was at follows: “I did like this visit not estimate if the learning that those results were not significant; enjoyment being with audio guide; otherwise, I could not talk with increased or not concerning the (Smileyometer) in Group A (MD=5) did not differ my friends”; “I would rather do a visit with an audio type of tour attended. significantly from Group B (MD=5), U=77, with small guide rather this one because the audio guide is new effect size (R=-0.02). Regarding the Learning, the stuff!”; “I would like to try with the audio guide”; “I difference between the knowledge acquired in the liked this visit with the lady explaining”. Group B after the exhibition without audio guides Summary of statistical Contribution - Human Tour Guide: the human tour guide should tests applied In order to explore the benefits of using audio guides tailor their tour by incorporating audio guides. We instead of human tour guides to foster children’s recommend having the human guide to capture the engagement and learning outcomes during their visit to attention of the children and allow them to listen to the Again-Again Table a natural science museum, we argue merging the use podcasts at the same time. The human tour guide Group Group Sig. (2- of audio guides along with human tour guides would be should direct the children to appropriate artifacts that tailed A B value) a better solution for providing more enjoyable learning are significant for the children and ask them to insert a Yes 84.6% 33.3% experiences. We propose to enhance the usability of specific code corresponding to the exhibit on the audio Chi-square the audio guides while adopting the benefits of human guide device. After everyone has gone through the Maybe 15.4% 25% p: 0.015 tour guides. Thus, allowing museums’ curators to audible content, the human tour guide should sum up No 0% 41.7% improve the flow of the experience as well as possible the information and invite the children to share what consequent learning. We recommend the following they think of the exhibit in order to promote reasoning, improvements for an audio guide tour aimed at discussion, and socialization among the group Smileyometer children: Group Group A B Sig. (2-tailed value) Concluding remarks MD=5 MD=5 Mann-Whitney IQR=0 IQR=0 p: 0.932 - Device: the physical device for audio guides (at least The poster presents the study conducted at the Whale without touchpad as the one of the Whale Museum) Museum of Madeira Island. Given the increasing Learning should be built for children’s usage and not for both interest that mobile devices have taken as mediators of Group Group adults and children. We noticed significant constraints museum experiences, the study aimed to verify if audio Sig. (2-tailed value) A M= B M= feedback when inserting the numbers through the guides enhance engagement and learning in 9-10-year- 2.23 15.42 Unrelated t-test keyboard. old children. To that end, we involved a sample of 25 p: 0.254 SD= SD= children and divided them into two groups, one doing 28.28 28.01 - Sensors: the usage of sensors should not interrupt the visit with the audio guide and a human guide, and the flow of the experience. Children noticed that the the other one only without the audio guide. Data were podcasts were always being repeated if they remained gathered through different measurement tools, at the same place. An improvement could be to utilize pre/post questionnaires, and observations. A series of the device’s keyboard to dial the exhibits’ number to statistical tests, triangulated with qualitative data, led prompt the children with the audible content; this to the conclusion that audio guides enhanced avoids the repetition of content while providing the engagement due to its “novelty” effect but hindered ability replay a content at the child’s will. learning and social interaction. Although our preliminary results go in this direction, the small - Content: the content of the podcasts should prompt sample gathered was a limitation and further studies the children to request additional information about the are needed in order to clarify these results. exhibits from the human tour guide. Acknowledgments ARDITI, project number M14-20-09-5369-FSE-000001. References 9. Nina Simon. 2010. The Participatory Museum. 1. Stephen Bitgood. 2009. Museum Fatigue: A Critical Retrieved September 24, 2016 from Review. Visitor Studies 12, 2: 93–111. http://www.participatorymuseum.org/. 2. Mihaly Csikszentmihalyi. 1998. Finding Flow: The 10. Maree Kristen Stenglin. 2004. Packaging Psychology of Engagement with Everyday Life. curiosities : towards a grammar of three- Basic Books, New York. dimensional space. Retrieved September 24, 2016 from 3. John H. Falk and Lynn D. Dierking. 2000. Learning https://ses.library.usyd.edu.au/handle/2123/635. from Museums: Visitor Experiences and the Making of Meaning. AltaMira Press. 11. Loic Tallon and Kevin Walker, eds. 2008. Digital Technologies and the Museum Experience - 4. Roy Hawkey. 2004. Learning with Digital Handheld Guide and Other Media. AltaMira Press, Technologies in Museums, Science Centres and UK. Galleries. NESTA Futurelab Research. 5. Federica Mancini and César Carreras. 2010. Techno-society at the service of memory institutions: Web 2.0 in museums. Catalan Journal of Communication & Cultural Studies 2, 1: 59–76. 6. Don Norman. 2013. The Design of Everyday Things. Basic Books, New York. 7. Janet Read and Stuart Macfarlane. 2002. Endurability, Engagement and Expectations: Measuring Children’s Fun. Interaction Design and Children, Shaker Publishing, Shaker Publishing, 1– 23. 8. Boa Rhee and Yongsoon Choi. 2015. Using Mobile Technology for Enhancing Museum Experience : Case Studies of Museum Mobile Applications in S. Korea. International Journal of Multimedia and Ubiquitous Engineering 10, 6: 39–44.