=Paper=
{{Paper
|id=Vol-1352/paper1
|storemode=property
|title=Fail Better: Lessons Learned from a Formative Evaluation of Social Object Labels
|pdfUrl=https://ceur-ws.org/Vol-1352/paper1.pdf
|volume=Vol-1352
}}
==Fail Better: Lessons Learned from a Formative Evaluation of Social Object Labels==
https://ceur-ws.org/Vol-1352/paper1.pdf
Fail Better: Lessons Learned from a
Formative Evaluation of Social Object Labels
Marcus Winter1, Michael John Gorman2, Ian Brunswick2,
Danny Browne2, Derek Williams2, Fionn Kidney2
1 2
University of Brighton Science Gallery
Watts Building, Lewes Road The Naughton Institute, Pearse Street
Brighton BN2 4GJ, UK Trinity College, Dublin 2, Ireland
marcus.winter@brighton.ac.uk {firstname.lastname}@sciencegallery.com
ABSTRACT In contrast to other in-gallery annotation systems, which
This paper presents Social Object Labels as an in-gallery typically aim for deep integration with a museum's
commenting platform and reports on a formative evaluation workflow and IT systems [e.g. 13,24], they provide a self-
of the concept focusing on visitors' awareness and mental contained layer of infrastructure and functionality that can
models of the developed prototype. Findings confirm many extend and complement existing technologies and
design assumptions underlying the evaluated prototype but engagement efforts in museums and galleries. This light-
also flag up serious problems resulting in low engagement weight approach makes SOLs quick and easy to deploy,
levels. They suggest a need to de-emphasise optical allowing for short-term trials in the target environment and
markers in the user interface, to provide visitors with a clear reducing costs and risks for organisations transitioning to
idea about the potential rewards of engagement and to align exhibit-based in-gallery commenting.
the interaction design with users' expectations shaped by the
wider interaction environment. The paper concludes with an SOLs are designed to be peripheral and unobtrusive in
tentative outlook on future design directions. order to not distract visitors' attention from an exhibit, but
conspicuous enough to be noticed by visitors and encourage
Author Keywords engagement. Getting this balance right is particularly
Social Object Labels; User Generated Content; Pervasive relevant in gallery environments, where curators try to
Displays; Ubiquitous Annotation; Mobile Interaction. create a certain atmosphere and visitor engagement with
exhibits often has contemplative undertones [26]. The SOL
ACM Classification Keywords prototype discussed here (Figure 1) addresses this design
H.5.m. Information interfaces and presentation (e.g., HCI): tension in two ways. Firstly, it uses small, passive,
Miscellaneous. monochrome e-ink displays that draw less attention than
active colour screens. Secondly, it delegates interaction for
INTRODUCTION creating, browsing and rating comments to the user's mobile
Social Object Labels (SOLs) are small interactive displays device, where it can be carried out in a discreet and
that can be attached to objects or places in order to collect personal manner. In order to account for different device
user-generated comments and ratings about them [30,31]. capabilities, personal preferences and varying degrees of
They typically display information about current digital literacy, the mobile interaction with SOLs can be
annotations for the object they are attached to, such as the initiated in various ways. The current design supports Near
number of comments or an average star rating, and support Field Communication (NFC), optical markers (QR codes)
interaction via a user's mobile device. In addition to the and manual input of a Web address (URL).
display component, which is deployed in-situ, the system
comprises a mobile web application for users to browse and
create content, an admin application to register SOLs to
physical objects and various backend services for content
moderation, analysis and syndication.
SOLs are currently being developed mainly in a museums
context to support visitors' social interpretation of exhibits.
8th International Workshop on Personalized Access To Cultural Heritage,
PATCH @ IUI 2015, March 29–April 1, 2015, Atlanta, USA.
Copyright held by the author(s). Figure 1. SOL prototype with e-ink display and NFC tag
There is a broad consensus in the literature about the need social object annotation. The system uses RFID technology
to evaluate ubiquitous computing technologies in the wild. to provide visitors with digital information and to anchor
Arguments include that important dimensions such as user-generated comments to exhibits. Visitors have the
device, space, people and time requires real use of a system option to additionally share posted comments on their
in authentic settings [1], that situated user behaviour is preferred social network. In order to protect the museum
fundamentally different from user behaviour in a lab as it and its visitors from inappropriate user-generated content,
puts more emphasis on improvisation and less emphasis on comments are pre-moderated, i.e. they are accessible to the
following a-priori plans [2] and that research into awareness public only after they have been approved by the museum.
and acceptance in particular requires real world settings
The most recent research in this context relates to the
where users behave more naturally [12, 15].
QRator [9] and Social Interpretation (SI) [4] projects. Both
This paper reports on the formative evaluation of SOLs at projects used similar technologies to explore social object
Science Gallery Dublin (SGD). It discusses the gallery annotation in museums. They involved both, touch screens
environment into which SOLs were deployed, the in the gallery space as a prominent way for curators to pose
methodology and findings of the formative evaluation and topical questions relating to the exhibition and collect
their design implications for SOLs and similar in-gallery visitors' responses via an on-screen keyboard, and QR
commenting systems. codes printed on object labels as a more peripheral
mechanism to collect visitors' comments about specific
BACKGROUND
Hawkey [11] points out that the goal of many museums is exhibits via a mobile application.
participation, which can take many forms, including simple Engagement statistics from the SI project found that while a
feedback, voting, collection of ideas and contributing to a large proportion of visitors used the more prominent touch
museum’s exhibits and interpretation. A key idea behind screens, only tiny minority scanned the OR codes next to
the drive for participatory museums experiences is that they exhibits. While in visitor interviews most respondents
provide visitors with opportunities to reflect and respond to claimed to have noticed the QR codes, only two out of ten
exhibits. Several efforts in the past have developed mobile staff reported to ever have observed visitors scanning a QR
technologies to support visitors' social interpretation of code [4]. This lack of engagement is further confirmed in a
exhibits while physically present in the gallery space. separate SI project evaluation [8], which found that despite
Van Loon et al. [28] present research around ARCHIE, a frequent iterations in the way QR codes were presented,
handheld guide with functionality to stimulate interaction they were ignored by a vast majority of visitors. The
with other visitors and the museum. Citing previous authors [8] identify several aspects contributing to the low
research [29], which shows that the use of handheld devices engagement with QR codes, including unreliable WiFi
in galleries can lead to isolated experiences and visitors connections, the requirement to install a proprietary mobile
paying less attention to the actual exhibits, they integrated app and a lack of appropriate framing by the museum that
communication, personalisation and localisation would explain their use and purpose to visitors.
functionality into a collaborative game to be played on Literature from the field of Human Computer Interaction
handheld devices in the gallery space. The game is (HCI) suggests more fundamental problems. Research into
anchored around exhibits, assigns players different roles users' perceptions, concerns and interaction with QR codes
and enables them to communicate via voice and other and RFID/NFC tags has revealed a wide range of
media, thereby promoting social interaction to support problems, including users having weak mental models of
visitors' intellectual, social, and cultural development. tags and tag interaction [10,17,19], feeling not in control
Seirafi and Seirafi [24] present FluxGuide, a commercial when interacting with tags [20] and being uncertain about
system for museums to present curated information on the security [20], integrity [19] and currency [10] of
mobile devices and enable visitors to add their own markers. Furthermore, market research studies [3,22] have
commentary about exhibits. The system deeply integrates identified low expectations of the potential rewards of
with a museum's IT backend to access digital materials scanning QR codes as the main reason non-engagement.
about exhibits, and to support social commenting and rating Many of these problems can be traced back to a lack of
of exhibits and related content by visitors. A key aspect information: markers with static signage cannot provide
discussed by the authors is that the system extends the dynamic information that could motivate users’ engagement
traditional one-way communication from museum to visitor and support their interaction. They require users to carry out
to a two-way model where information flows in both several interaction steps with their mobile phone before
directions and between users, thereby enabling new forms disclosing dynamic information on the device screen. SOLs
of interpretation and learning. address this point by showing up-to-date information in-situ
Hsu and Liao [13] describe a prototype mobile application next to the exhibit, i.e. independent of the visitor's mobile
at the Exploratorium science gallery in San Francisco device. The information is visible before interaction takes
integrating self-guided exploration of an exhibition with place and enables users to make a more informed decision
about their engagement. SOLs also provide a more coherent
user experience after the interaction took place: while
markers with static signage do not change appearance when
a user submits content, SOLs dynamically update their
display (e.g. increase the comment counter) and show a
physical trace of the interaction in the environment.
An important aspect of the evaluation reported on is to
assess how gallery visitors notice SOLs and interact with
them. Of particular interest is whether the provided
dynamic information about comments for an exhibit can
help to address the usability problems experienced with
static markers, support visitors to form a suitable mental
model of the tag interaction and overcome low expectations
of the potential rewards of engaging with markers.
GALLERY DEDPLOYMENT
Two SOL prototypes were deployed at Science Gallery
Dublin (SGD) during the recent Fail Better exhibition,
which explored the instructive role of failure in stimulating
creativity in research and development. The exhibition ran
for 12 weeks (7 February to 27 April 2014) and attracted
92,000 visitors during that time. The two displays were
installed during the final two weeks of the exhibition.
Figure 3. SOL (circled red) installed next to the exhibit
Drawing on the idea of social objects [7,25], the prototypes Apparatus for Facilitating the Birth of a Child by Centrifugal
were attached to exhibits that are likely to provoke a Force on the first-floor gallery space.
reaction from visitors. One display (SOL1) was installed on
the ground floor next to Superman's Wheelchair (Figure 2) Framing
and a second display (SOL2) was installed on the first floor While Giannachi and Tolmie [8] suggest that commenting
next to the Apparatus for Facilitating the Birth of a Child mechanisms involving novel technologies such as QR
by Centrifugal Force (Figure 3). Both exhibits were key codes need suitable framing in order to give visitors a clear
pieces of the exhibition and attracted much interest from reason why to scan the code and why to share their
visitors. thoughts, no notices, instructions or calls to action were
provided in the gallery space to explain the purpose or use
of SOLs. Instead, it was hoped that the dynamic
information shown on SOLs would provide enough clues
for visitors to guess their purpose and at the same time
promote engagement by piquing their interest. The
deployment therefore relied on the participatory
environment in SGD, which invites visitor engagement in
many different ways, and on visitors being inquisitive and
tech-savvy enough to try out the installed SOLs.
Physical integration
In accordance with Kules et al. [16], who suggest that
interactive installations should be situated in locations with
a sustained flow of people and sufficient space for
interaction, both exhibits had enough space for visitors to
walk around them and to approach the SOL (Figures 2, 3).
Considering that displays installed at eye height and close
to other eye-catching objects receive more attention [14],
SOL1 was installed at eye height next to the object label
(Figure 4a) while SOL2 was installed close to eye height
just below the object label (Figure 4b). Both displays were
easy to read and to scan, taking into account that the height
Figure 2. SOL (circled red) installed next to the exhibit of physical markers directly impacts on the success and
Superman's Wheelchair on the ground-floor gallery space. ease of interaction [10]
In addition to these moderated engagement opportunities,
where visitors could discuss exhibits and related issues with
a student-mediator, Fail Better had various interactive
installations that invited people to contribute their views
and explore additional information.
a) b)
Figure 4. SOLs were installed close to eye-height a) next to
the exhibits' object label or b) below the object label.
Technical integration
In parallel to the free public WiFi offered by SGD to all
visitors, SOLs were connected to a separate staff network
available in the gallery space. Reflecting variations in the
WiFi signal strength, SOL1 on the ground floor had a very
solid connection while SOL2 on the first floor sometimes a) b)
had connection problems, e.g. when re-connecting after Figure 5. Twitter printer (a) in the ground floor gallery (b).
having been charged.
A Twitter printer (Figure 5) was installed in the ground
While it was initially planned to run SOLs on mains power floor exhibition space. Consisting of a small thermal
to reduce maintenance, the actual deployment was battery transfer printer connected via Bluetooth to a hidden
operated and depended on gallery staff to periodically computer, the device uses the Twitter API to periodically
check and recharge the displays. Once flat, SOLs were search for Tweets containing the exhibition hashtag
taken from their casing, charged for 3 hours and then re- (#failbetter) and then prints them out. The resulting endless
inserted into the casing, which occasionally left an empty print roll spools down from the printer and is collected in a
casing in the exhibition space for several hours. box on the floor below the printer.
Information environment
Brewer [5] introduces the notion of an information
environment as a way to describe how the information
shown by an ambient display integrates with other
information available at the site. While the evaluated SOL
prototype is capable to display information about the
exhibit it is attached to, such as a title, description or
image, the installed units were configured to only show the
number of comments for an exhibit, a QR code and an URL
for visitors to connect their mobile phone (in addition to the
NFC tag integrated into the casing). Consequently, there
was no information overlap between SOL and object labels
or other information available in the gallery space.
Interaction environment
A standard feature in all exhibitions at SGD are student Figure 6. Fail Wall in the first floor gallery.
mediators with knowledge of the relevant subject area,
A Fail Wall (Figure 6) was installed in the first-floor
ready to answer questions about exhibits and involve
gallery space. The installation prompted visitors to write a
visitors into discussions about related issues. Another
personal failure on a plastic tile, photograph it for upload to
common feature are opportunities for visitors to get
an online photo stream and then put up the physical tile on
involved in research studies by taking part in a short
the wall for other visitors to read. The installation involved
experiment or filling in a questionnaire. For instance,
a work area for writing on the plastic tiles, a camera area
visitors to Fail Better had the opportunity to take part in an
where the plastic tiles could be photographed and a display
experiment run by the School of Psychology at Trinity
area where tiles could be put up on simple shelves. A
College Dublin, which examined attitudes to failure and
separate second display area was used to feature a Fail of
their impact on wellbeing.
the Day selected and put up by gallery staff.
being studied would reasonably expect to be observed by
strangers, with reference to local cultural values and to the
privacy of persons who, even while in a public space, may
believe they are unobserved." [6] (p.13).
Observations focused on "encounters" as a quantifiable
unit. Encounters were conceptualised as situations where
visitors had a clear chance to notice and engage with a
SOL. At minimum, an encounter involves a visitor
approaching and stopping at an exhibit. Visitors might then
look at the exhibit, read the object label, look at the SOL,
point others to the SOL or engage with the SOL in various
ways. Observations were coded on the spot using a coding
template supporting both quantitative and qualitative
observations.
Figure 7. Touch screen with additional video footage. The observations were carried out over two days during the
last week of the exhibition. Total observation time was 6
Some exhibits had associated projections or interactive hours and 35 minutes, during which 212 encounters were
touch screens that offered additional information. For observed. Of these, 90 involved Superman's Wheelchair
instance, Superman's Wheelchair had a small (10 inch) and 122 the Apparatus for Facilitating the Birth of a Child
touch screen installed in front of the exhibit showing a by Centrifugal Force. The observations suggest that 62
selection of related video footage (Figure 7). (29%) visitors were on their own, 82 (39%) part of a couple
Together, these engagement opportunities offered analogue and 68 (32%) part of a group of three or more.
and digital routes to participation, spanning a wide range of
modalities, capabilities and learning styles. Rounding off
the open and participatory atmosphere, SGD offers free
wireless internet access to visitors and has no restrictions on
mobile phone use in the gallery space, enabling visitors to
take pictures of exhibits and share their experience live on
social networks.
EVALUATION
The formative evaluation focused primarily on qualitative
aspects such as visitors' awareness and mental models of Figure 8. Observed visitor attention during encounters.
SOLs. It involved visitor observations and structured
interviews carried out in the gallery space. The study In the observed 212 encounters, 202 visitors (95%) looked
employed convenience sampling that includes gallery at the exhibit (some only read the label), 181 visitors (85%)
visitors most easily observed and willing to take part in an read part or all of the object label and 35 visitors (17%)
interview. However, in order to maximise the range of visibly noticed the SOL (Figure 8).
views and insights, sampling was still informed by basic Of the 35 visitors who looked the SOL, six (17%) touched
strategies from probability sampling to address potential the NFC tag in a manner one would press a button, four
biases. With respect to coverage, the study was carried out (11%) touched the screen to see if it was interactive, three
over a Friday (workday) and Saturday (weekend), which (9%) scanned the QR code and two (3%) scanned the NFC
are likely to vary in audience volume and composition. tag with their mobile phone (Figure 9).
With respect to visitor sampling, the researcher aimed for a
balanced demographic and included people visiting on their
own and in pairs or groups.
Visitor observations
Observations were carried out in the gallery space to find
out whether visitors notice SOLs and how they interact with
them. In order to not disturb people’s natural behaviour,
observations were carried out without prior notice or
informed consent. The researcher's conduct during these
observations was informed by the British Psychological
Society's code of ethics and conduct, restricting Figure 9. Observed SOL attention and interaction.
observations "[...] to those situations in which persons
The observations suggest that of the 35 visitors who visibly out by the interviewer, nine visitors (53%) answered yes
noticed the SOL, many assumed that it was somehow and eight visitors (47%) answered no (Figure 10).
interactive but were not sure how to actually interact with
it. At least ten (28%) assumed a direct interaction model
and tried to press the NFC tag or touch the display screen.
When these actions had no effect, visitors did not further
investigate but simply moved on.
With regard to group dynamics, two out of the three
observed QR code scans were carried out by visitors who
were part of a couple or group. In both cases the primary
actor tried to get their partner or other group members
involved, either by pointing out the SOL to them or by
sharing their mobile screen after scanning to read submitted Figure 11. Interviewees' assumptions what SOLs are for
comments. In none of the observed interactions did visitors
actually contribute a comment. With regard to visitors' understanding of the purpose of
SOLs, 14 visitors (82%) thought they were for reading
Visitor interviews comments, 12 visitors (71%) thought they would also allow
In addition to observations, which focused on visitors' submitting comments and three visitors (18%) were not
awareness and interaction, structured interviews were sure (Figure 11). This suggests that the majority of visitors
carried out to understand visitors' mental models of SOLs made correct assumptions about the purpose of SOLs in the
and explore their motivations and barriers to engagement. gallery space once they were aware of them.
The interviews also included questions to further qualify
recorded observations.
Visitors were approached by the researcher after their
encounter with the exhibit and possibly the SOL. Visitors
were informed about the research context and asked to sign
a consent form before the interview. Interviews lasted
between 5-7 minutes and followed a fixed structure. The
interviewer recorded answers in a coding template. A short
section with demographic questions was filled in by
participants themselves after the interview.
Figure 12. Interviewees' assumptions how SOLs work.
A total of 17 visitor interviews were carried out, involving
ten female (59%) and seven male (41%) participants. The Asked about interactivity, 14 respondents (82%) thought
age range of interviewees reflects SGD's target audience that SOLs were interactive in some way while one visitor
with ten participants between 25-34 years of age (59%), (6%) thought they were not interactive and two visitors
three between 16-24 years (18%) and others falling in equal (12%) were not sure. When asked how the interaction with
measure into older age brackets. All participants reported to SOLs might work, 13 visitors (76%) answered that one
own a smartphone (defined as a mobile phone with internet would scan the QR code, two visitors (12%) thought it was
access and touch screen) and more than half of all a touch screen and another two visitors (12%) were not sure
interviewees (53%) indicated that they had scanned a QR (Figure 12). Contrary to observations, which suggest that
code before. None had ever have scanned an NFC tag. many visitors assumed a direct interaction model, the
interviews indicate that most visitors understood that
scanning the QR code was the primary mode of interaction.
Figure 10. Interviewees' self-reported awareness of SOLs .
When asked whether they had noticed the SOL, referred to
as "the small display next to the object label" and pointed Figure 13. Interviewees' expectations of content.
When asked what kind of content they would expect when low expectations of QR codes are well documented [3,22],
scanning the SOL, eight visitors (47%) answered they and ignoring them in this manner would be in line with the
would expect to be directed to the gallery's website where known phenomenon of "display blindness" [18], where
they could read and submit comments, six visitors (35%) people have such low expectations of displays in their
would expect a list of comments and three visitors (18%) environment that they automatically blank them out.
would expect "information" but did not further specify what
This interpretation is further supported by the low numbers
kind of information (Figure 13).
of actual scans, and by visitor interviews suggesting a
weary and sometimes negative attitude towards QR codes.
While most visitors understood that they had to scan the QR
code on the SOL with their mobile phone, they had no clear
expectations what to expect in return. Many visitors
reported that a lack of interest in the expected content was
their main barrier to engagement. This suggests that the
dynamic information provided by the current SOL
prototype is not enough to give visitors a clear idea about
the content they can expect and motivate engagement.
Possible ways to address these problems include de-
Figure 14. Interviewees' barriers to engagement. emphasising the QR code in the user interface and
providing more detailed information on the display. Current
Finally, when asked what it would take for them to engage practice in museums (and elsewhere) is to use QR codes not
with the SOL and what was holding them back, six visitors only as an optical marker to be scanned by mobile devices,
(35%) mentioned a lack of interest in comments or more but also to advertise interaction opportunities to potential
generally in further engaging with the exhibit, four visitors audiences. While the former is a robust, cheap and
(24%) mentioned technological barriers such as not having relatively well-known mechanism, the latter has come into
a QR code scanner installed on their phone or not being disrepute due to the low quality of content often linked to.
sure exactly how to scan a code, two visitors (12%) Displaying QR codes less prominently or relegating them to
mentioned a lack of information in the gallery that would a secondary screen together with other connection options
explain the purpose and use of SOLs while five visitors helps to split these two roles and make use of their qualities
(29%) did not answer the question (Figure 14). as robust and well-supported machine-readable markers
while not misusing them to advertise interactivity to
DISCUSSION potential audiences. At the same time, users’ lack of interest
The primary focus of this formative evaluation has been on in the expected content can possibly be addressed by
visitors' awareness and mental models of SOLs. It providing more engaging information on the SOL. This
employed observations to find out whether visitors notice could involve “bringing the data forward” [23] by showing
SOLs and how they engage with them, and visitor some actual comments on the display that can give visitors
interviews to compare observed awareness with self- a better idea of potential rewards of engagement.
reported awareness and further probe visitors’
understanding of the purpose and use of SOLs. Another recurring theme in this evaluation is the need to
support direct interaction on SOLs without a mobile device.
Visitors' awareness of SOLs directly relates to the inherent Visitors' expectations of how to interact with SOLs are
design tension between being peripheral and not distract shaped by the wider interaction environment and their
from the exhibit but noticeable enough to encourage experience with other interactive installations, which often
engagement. With observations suggesting that only 17% of support hands-on direct interaction in the form of touch
visitors are aware of SOLs and self-reported awareness at screens or buttons. Observations suggest that visitors build
53%, the current SOL design is clearly not too obtrusive or on this experience when trying to figure out how to interact
distracting. In fact these numbers suggest that SOLs could with SOLs, e.g. by tentatively touching the display or trying
be more conspicuous to reach higher levels of awareness to press the NFC tag like a button. Supporting direct
without diverting too much attention from the exhibits. interaction is also desirable from a curatorial perspective as
The strong discrepancy between observed and self-reported it is more inclusive and enables visitors without
awareness is remarkable. While some of this difference smartphones and technical skills to participate.
might be attributed to the Hawthorn Effect [27] or "good Two common barriers to engagement mentioned in visitor
bunny effect" [21], where respondents try to give "the right" interviews are a lack of information about SOLs and
answer to a researcher's questions, another possible technological issues, such as not having a QR code scanner
interpretation is that visitors automatically blank out QR installed. With regard to the former, the results refute the
codes when they see them, resulting in only a passing notion of SOLs being self-explanatory and the displayed
glance that is difficult to detect in observations. People's dynamic information being intriguing enough to encourage
engagement. Instead, future deployments should learn from SOLs, were unclear exactly what kind of content to expect
others [8] and provide information that frames and explains and seemed to assume a direct interaction model in line
their purpose and use to visitors. With regard to the latter, with other interactive installations in the exhibition. Various
the results suggest that it is beneficial to support a wide opportunities were identified how the information
range of mechanisms to connect a mobile device and presentation and overall design of SOLs could be improved.
thereby minimise the chances of technical issues or
One key finding relevant beyond the immediate context of
preferences becoming real barriers to engagement. In
SOLs is the observed QR code blindness. Visitors seem to
addition to the currently supported NFC tags, QR codes and
blank out QR codes in their environment much in the same
manual URL input, this could, for instance, involve posting
way as display blindness has been observed for people's
comments on SOLs via commonly used social platforms
attention to public displays [18]. In both cases low
like Twitter, which at least a subset of visitors are familiar
expectations based on previous poor experiences can be
with and have already installed on their mobile device.
identified as the main reason. The paper suggests to
decouple the technical utility of QR codes as markers for
SUMMARY AND CONCLUSIONS
This paper has presented SOLs as a platform for the social mobile interaction from their misguided use as a means to
interpretation of exhibits in the gallery space and reported advertise interactivity. This can be achieved by making QR
on a formative evaluation in the field. codes less prominent and showing them as one of many
options for mobile interaction.
Drawing on literature from museum studies and HCI, it has
discussed a range of problems relating to visitors' Overall the evaluation resulted in valuable insights how
awareness, acceptance and engagement with in-gallery people perceive and understand SOLs in a gallery space,
commenting systems using optical or radio-frequency confirming that ubiquitous computing technologies should
markers for mobile interaction. The paper has linked these be evaluated in realistic settings. The findings will help to
problems to a lack of dynamic information that could further develop the current prototype in order to improve
motivate and support user interaction, and discussed how the user experience. It is envisaged that the next design
SOLs aim to address this by providing current information iteration will depart from a single-screen interaction model
about object annotations in-situ and independent from the with sparse information upfront and instead move to a
mobile device screen. multi-screen model with direct interaction and more
detailed information upfront. It is hoped that this change
Motivating the empirical evaluation of SOLs at SGD with will raise awareness levels by avoiding the effects of QR
the need to evaluate ubiquitous computing technologies in code blindness, increase engagement by presenting more
realistic contexts, the paper described in detail the gallery relevant content and better align with people's expectations
environment into which SOLs were deployed. It explained shaped by the wider interaction environment.
the (lack of) framing, the physical integration with the
exhibition, the technical integration with regard to ACKNOWLEDGMENTS
networking and power supply, the information environment We thank all the visitors and student mediators at Science
and the interaction environment in the gallery space. The Gallery Dublin who took part in or otherwise supported this
latter was described as rich in engagement opportunities, evaluation. We also would like to thank the reviewers of
with analogue and digital routes to participation and this paper for their insightful comments and feedback.
addressing a wide range of modalities, capabilities and
learning styles. While such an environment primes visitors REFERENCES
for engagement it also is very competitive, requiring 1. Abowd, G. and Mynatt, E. Charting past, present, and
installations to provide an excellent user experience in order future research in ubiquitous computing. ACM Trans.
to attract visitors’ attention and at the same time conform to on Computer-Human Interaction, 7(1), (2000), 29–58.
a common set of interaction mechanisms as visitors’
2. Abowd, G. What next, ubicomp? Celebrating an
expectations are shaped by their experience with other
intellectual disappearing act. In Proc. UbiComp 2012,
interactive installations in the exhibition.
1–10.
The formative evaluation was based on visitor observations
3. Aguirre, D., Johnston, B. and Kohn, L. QR Codes Go to
and structured interviews to assess visitors' awareness and
College. Archrival (2011),
mental models of SOLs. While engagement numbers for the
http://www.archrival.com/ideas/13/
deployed SOLs were disappointing, the evaluation flagged
up a number of problems with the current design and can 4. Bagnal, G., Light, B., Crawford, G., Gosling, V.,
therefore be seen as an instructive failure. The evaluation Rushton, C. and Peterson, T. The Imperial War
results suggest that the majority of visitors understands in Museum's Social Interpretation Project. Digital R&D
principal that SOLs are for reading and writing comments Fund for the Arts. http://usir.salford.ac.uk/29146/1/
and that they can be accessed by scanning a QR code. Academic_report_ Social_Interpretation2.pdf.
However, many visitors were unaware of the deployed
5. Brewer, J. Factors In Designing Effective Ambient Field Study on User Perceptions. In Proc. CHI 2007,
Displays. In Proc. UbiComp 2004, pp. 1–2. 991–994
6. British Psychological Society (2011). Code of Ethics 18. Müller, J., Wilmsmann, D. and Exeler, J. Display
and Conduct. blindness: The effect of expectations on attention
http://www.bps.org.uk/system/files/documents/code_of_ towards digital signage. In Proc. Pervasive 2009,
ethics_and_conduct.pdf. 19. Neill, E. O., Thompson, P., Garzonis, S. and Warr, A.
7. Engeström, J. Why some social network services work Reach out and touch: using NFC and 2D barcodes for
and others don’t - Or: the case for object-centered service discovery and interaction with mobile devices.
sociality. (2005). http://www.zengestrom.com/blog/ In Proc. Pervasive 2007, 19–36
2005/04/why-some-social-network-services-work-and- 20. Riekki, J., Salminen, T. and Alakärppä, I. Requesting
others-dont-or-the-case-for-object-centered- Pervasive Services by Touching RFID Tags. IEEE
sociality.html. Pervasive Computing, 5(1), (2006), 40–46.
8. Giannachi, G. and Tolmie, P. Info-Objects: Embedding 21. Robson, C. Real World Research. (2002) Oxford, UK:
Objects with Audience Interpretation. Research Report. Blackwell Publishing
Digital R&D Fund for the Arts, (2012)
http://www.nesta.org.uk/library/documents/Academic_r 22. Russel Herder. The QR Question: Are QR codes an
eport_Social_Interpretation-1.pdf. effective marketing tool for engaging customers? (2011)
http://www.russellherder.com/ wp-content/themes/
9. Gray, S., Ross, C., Hudson-Smith, A. and Warwick, C. RH_Theme_2011/pdf /QRQuestion_Whitepaper.pdf.
Enhancing Museum Narratives with the QRator Project:
a Tasmanian devil, a Platypus and a Dead Man in a Box. 23. Saffer, D. Microinteractions. (2013). Sebastopol, CA:
In Proc. Museums and the Web 2012. O’Reilly.
10. Hardy, R., Rukzio, E., Holleis, P. and Wagner, M. 24. Seirafi, A. and Seirafi, K. Fluxguide: Mobile
Mobile interaction with static and dynamic NFC-based Computing, Social-Web & Participation @ the Museum.
displays. In Proc. MobileHCI 2010, 123-133. In Proc. Forum Medientechnik 2011, 325-331
11. Hawkey, R. Learning with Digital Technologies in 25. Simon, N. The participatory museum. (2010).
Museums, Science Centres and Galleries. Futurelab, http://www.participatorymuseum.org/
Bristol, Report 9, (2004). http://archive.futurelab.org.uk 26. Tröndle, M. and Wintzerith, S. A museum for the
/resources/publications-reports-articles/literature- twenty-first century: the influence of “sociality”on art
reviews/Literature-Review205. reception in museum space. Museum Management and
12. Hazlewood, W. R. and Coyle, L. On Ambient Curatorship, 27(5), (2012), 461–486.
Information Systems. Int. Journal of Ambient 27. Turnock, C., and Gibson, V. Validity in action research:
Computing and Intelligence, 1(2), (2009), 1–12. a discussion on theoretical and practice issues
13. Hsu, H. and Liao, H. A mobile RFID-based tour system encountered whilst using observation to collect data.
with instant microblogging. Journal of Computer Journal of Advanced Nursing, 36 (3), (2001), 471-477.
Systems Science, 77(4), (2011), 720–727 28. Van Loon, H., Gabriëls, K., Teunkens, D., Robert, K.,
14. Huang, E., Koster, A. and Borchers, J. Overcoming Luyten, K., Coninx, K., & Manshoven, E. Designing for
assumptions and uncovering practices: When does the interaction: socially-aware museum handheld guides. In
public really look at public displays? In Proc. Pervasive Proc. NODEM 06, (2006).
2008, 228–243 29. Vom Lehn, D. and Heath, C. Accounting for New
15. Ju, W. and Sirkin, D. Animate Objects: How Physical Technology in Museum Exhibitions. International
Motion Encourages Public Interaction. In Proc. Journal of Arts Management, 7(3), (2005), pp. 11-21
Persuasive 2010, 40–51. 30. Winter, M. Inch-scale Interactive Displays for Social
16. Kules, B., Kang, H., Plaisant, C. and Rose, A. Object Annotation. In Adj. Proc. UbiComp 2013, 183-
Immediate usability: a case study of public access 186.
design for a community photo library. Interacting with 31. Winter, M. Social Object Labels: Supporting Social
Computers, 16(6), (2004), 1171 – 1193 Object Annotation with Small Pervasive Displays. In
17. Mäkelä, K., Belt, S., Greenblatt, D. and Häkkilä, J. Proc. PerCom 2014, 489-494.
Mobile Interaction with Visual and RFID Tags – A