The Augmented Shopping Trolley: An Ambient Display To Provide Shoppers with Non-Obvious Product Information Jon Bird, Vaiva Kalnikaité and Yvonne Rogers Pervasive Interaction Lab The Open University Milton Keynes, MK7 6AA, UK {j.bird, y.rogers}@open.ac.uk, vaivak@gmail.com ABSTRACT information about the global consequences of their The Augmented Shopping Trolley consists of an ambient consumer decisions [2]. Our goal is to provide ‘non- handlebar display connected to a scanner. When a shopper obvious’ nutritional, ethical and environmental product scans an item the handlebar lights up to provide them with information, that is, information that is not immediately information about the product, such as its nutritional, obvious from an item’s packaging or label, in a form that is ethical or environmental attributes, that are not obvious as salient as the features that typically inform consumers’ from its packaging or label. The system is designed to decision making. The Augmented Shopping Trolley (Figure seamlessly integrate with a shopping experience: it uses 1) is designed so that it fits as seamlessly as possible into a familiar supermarket technologies; it keeps both of a supermarket shopping experience. We use familiar shopper’s hands free; and the simple ambient display supermarket technologies: augmenting a standard shopping facilitates the ‘fast and frugal’ decision-making typically trolley by attaching a scanner and embedding an ambient observed in a supermarket. Our initial lab-based study display in the handlebar. This gives our system two shows that the display can be understood at a glance and advantages over using mobile devices to provide product used to select items based on a product’s nominal properties information. First, the trolley scanning technology is faster (for example, it is organic), ordinal properties (for example, [4] and second, because the ambient display is built into the it has low, medium or high food miles), as well as a trolley handlebar a customer’s shopping experience is not combination of the two at the same time. Where as usability disrupted by having to repeatedly access and store a mobile was the focus of our initial design, ethical issues have come display. Underhill [10, see chapter 4] emphasizes the to the fore as we develop the system for use in importance of having both hands free during shopping. supermarkets and we discuss how these are influencing our design. Author Keywords Persuasive technologies, ambient display, shopping, product information, ethics. ACM Classification Keywords H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION In a supermarket, shoppers tend to make snap judgments based on just a few salient cues (low price, recognized brand and attractive packaging) and they rarely take time to read product information labels [7]. However, recent Figure 1. The Augmented Shopping Trolley display consists of consumer surveys indicate that shoppers want more 16 LEDs embedded in the handlebar, each of which can be set to green, red or orange Permission to make digital or hard copies of all or part of this work for Our approach to designing an effective ambient display, personal or classroom use is granted without fee provided that copies are first outlined in [9], is motivated by studies of ecological not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, rationality which investigate how people make reasonable or republish, to post on servers or to redistribute to lists, requires prior decisions given the constraints of limited time, information specific permission and/or a fee. and computational resources that characterize most real Copyright © 2011 for the individual papers by the papers' authors. world situations [6, 8]. This research indicates that most Copying permitted only for private and academic purposes. This volume is published and copyrighted by the editors of PINC2011.. natural decision making is made on the basis of ‘fast and (idle state) to a half second sweeping movement of orange frugal’ heuristics – short-cut strategies where people ignore that indicates scanning is in progress. There is then a beep, most of the available data and instead focus on the most as typically heard at a checkout counter, to signal that useful information and process it quickly. Often people scanning is completed and the display then changes to a make a decision based on a single reason as this strategy is new state that provides relevant information about the quick and simple and avoids having to weigh up trade-offs product. If the display is configured to show a nominal between multiple and potentially conflicting options. This property of the product, then it flashes green if the property approach is not rational in certain environments, namely, is present and shows the idle state if it is not. If the display those where available pieces of information are is providing ordinal information about the product, the approximately equally useful. However, in a shopping display employs a bar graph metaphor, with the number of environment, the distribution of information usefulness is red pixels indicating the degree to which an item has a highly skewed, that is, the most useful piece of information property. Specifically, if an item has a low degree of a is a lot more important than the second most useful, which property then pixels 1-3 turn red and 4-16 turn green; if in turn is considerably more important than the third, etc. medium then pixels 1-8 turn red and pixels 9-16 turn green; Our handlebar ambient display consists of just sixteen and pixels 1–13 turn red and 14-16 turn green if the item LEDs. When a shopper scans a product, a few pieces of has a high degree of a particular property. Finally, both non-obvious information, such as whether it contains nuts, these representations can be combined to show the value of is fair trade or has low food miles, are displayed as a salient a nominal and an ordinal property at the same time. In our pattern on the display. study, after a participant selected or discarded an item, the display changed back to the all green idle state. Given that information salience influences a person’s behaviour unconsciously [1], rather than through rational LAB-BASED SYSTEM EVALUATION reflection, this raises ethical concerns about the Augmented 5 adults (1 female, 4 male, aged between 20 and 40) took Shopping Trolley, chief of which is that this system could part in a lab-based evaluation of the Augmented Shopping potentially manipulate people into behaving in ways that Trolley. Each participant completed 12 shopping scenarios they would not otherwise do, and furthermore, that they where they were asked to pick up and scan 5 items of a might not be aware that they had been manipulated. This particular product type and only select those items that met concern, and also issues to do with privacy and clarifying specified criteria. A scanner was attached to the shopping how our system benefits shoppers, form the ethical trolley (Figure 1) but was non-functional and the handlebar considerations that are influencing how we deploy the display was changed using a Wizard of Oz methodology. Augmented Shopping Trolley in a supermarket. On the basis of the changes in the patterns on the handlebar The paper is structured as follows: first, we describe the display, participants had to decide whether to select the display hardware and how it conveys product information; item and place it in their trolley or discard it and place it on second, we describe a lab-based evaluation of the system an adjacent table. Since this was an exploratory study, we that demonstrates the efficacy of the ambient handlebar were intentionally vague about the operation of the ambient display for conveying non-obvious product information; display as we wanted to see whether participants could and third, we describe the ethical issues that are informing understand it intuitively. We only told participants that the the development of the system for use in supermarkets. display patterns would change depending on whether a product had a specific property (yes/no), the degree to AMBIENT HANDLEBAR DISPLAY DESIGN which a product had some property (high/medium/low) or a The handlebar display was designed to provide shoppers combination of the two. Participants were allowed to scan with salient and easy to read information about a scanned the items as many times as they wanted and in any order, product’s nominal properties (for example, whether it is before they made their decision about whether to select a organic or contains nuts), its ordinal properties (for particular item. We used 4 product types: milk; breakfast example, if it has low, medium or high food miles), as well cereal; wine; and juice. Each shopping scenario used one of as a combination of the two at the same time. We the product types and participants were asked to select from constructed the display by attaching 16 bicolour LED units 5 different items. For example, select those bottles of wines to a piece of wood inside a transparent plastic tube (Figure that meet the specified criterion (fair trade) and put them in 1). This replaced the plastic handlebar in a standard the trolley, and place the others on the discarded items shopping trolley. The LEDs are controlled using 2 table. Each of the items was a real product but we masked TLC5940 chips (Texas Instruments) that are driven by an any product information on the packaging and told Arduino microcontroller. In our lab-based study this is participants to only use the handlebar display to decide attached via a USB cable to a laptop running a Processing whether they should select an item or not. The experimenter application. Each LED unit can be set to red, green or playing the Wizard of Oz role sat at a table on which the 20 orange (when both the green and red LEDs are on). Each shopping items were grouped by product type. Each item time a product is scanned, the display changes in the was individually numbered so that the experimenter could following way. First, it goes from an all green background change the display appropriately when the participants use and were able to quickly read it even though they were scanned a particular item. not given explicit information on the meaning of the display patterns. Only two participants scanned items more than In the first 4 shopping scenarios the handlebar display once and this was exploratory activity at the beginning of indicated whether a scanned item had a particular nominal the evaluation when they were seeing how the interface property or not: whether a milk product was organic; worked. whether a breakfast cereal contained nuts; whether a bottle of wine was fair trade; and whether a carton of juice ETHICAL ISSUES AND FURTHER DEVELOPMENT contained added sugar. In 2 of these scenarios the Whereas usability issues informed our initial design, ethical participants had to select items that had a particular considerations are shaping the development of the property and in the other half they had to discard items if Augmented Shopping Trolley for use in supermarkets. This they had a particular property. For example, in the first is because our ambient display not only provides salient shopping scenario participants had to select a milk product product information for shoppers, but also potentially if it was organic and discard it if it was non-organic; in the influences what they purchase. The use of persuasive second shopping scenario participants had to select a technologies raises ethical concerns for many people. For breakfast cereal if it did not contain nuts and discard it if it example, Page and Kray [3] used an online questionnaire to did. investigate people’s views on the ethics of using persuasive In the next stage of the evaluation, the participants technologies to encourage healthy living. 72 participants completed 4 shopping scenarios where the display indicated rated the ethical acceptability of a number of different whether a product contained a low, medium or high value scenarios which varied in 3 different factors: whether a of a particular ordinal property. The task was to select items participant chose to use the technology or an external that had a specified property to a particular degree. agency initiated its use; whether there was a clear benefit Specifically, participants were asked to select milk with a for the participant or not; and the technology used (text medium fat content, cereals with a high sugar content, wine messages to the participant’s mobile phone; public with low food miles and juice with a medium water content. announcements in the participant’s location; Facebook In none of these scenarios were participants asked to messages; restrictions on the participant’s bank account; discard items if they had properties of a particular degree. and electric shocks). The results indicated that the majority The final 4 shopping scenarios tested whether participants of the participants viewed the use of persuasive could understand the display when it simultaneously technologies in most of the questionnaire scenarios as showed information about both a nominal and an ordinal unethical. When there was no clear benefit to the property of a scanned item. Participants were asked to participant, mobile phone were considered the most ethical select milk that was organic and low fat, cereals that persuasive technology. However, approximately the same contained nuts and had a medium sugar content, juice that proportion of participants (40%) considered them very had added sugar and high water content and wine that was ethical or ethical as the proportion that considered very not fair trade and had medium food miles. Only in the wine unethical or unethical when. A large majority of scenario did participants have to reject items on the basis of participants found the other technologies very unethical or information about a nominal property of the product. unethical. In scenarios where the use of a technology would clearly benefit the participant, for example, save their life, USABILITY RESULTS then this usage was considered slightly more ethical than 4 out of the 5 participants were able to interpret the ambient the cases where the technology did not benefit the handlebar display and complete all the tasks without any participant. However, it is not clear whether these mistakes. The other participant made one consistent error in differences were statistically significant. When people were 2 of the first shopping scenarios where the task was to able to freely choose whether to use a persuasive discard items if they had a particular nominal property: they technology or not, then texts, public announcements and selected, rather than discarded, them, but did not repeat this Facebook messages were considered ethical by the majority error in the final shopping scenario which also required an of respondents, in comparison to the situation where the use item to be discarded if it had a particular nominal property. of the persuasive technology was initiated by an external Several participants reported that they found the tasks entity (for example, the UK’s National Health Service). where they had to discard items with particular properties Electric shocks and bank account restrictions were more difficult and it did seem to increase the cognitive load considered very unethical or unethical by the majority of in all participants, resulting in a slightly slower response respondents, even when a participant chose to use them. time (approximately 2 seconds, rather than 1 second for the Page and Kray’s findings seem to concur with a central other conditions). This could be due to the colours used in factor identified by applied philosophical analyses of the display: a nominal property is indicated by a green ethical behaviour, for example, the use of persuasion in blinking display, a colour that many people associate with advertising [5]. Namely, the ethics of an action are positive properties, rather than ones that should be avoided. determined, to a large degree, by the extent to which that All participants reported that the display was intuitive to action impacts on an individual’s autonomy, that is, their capacity to choose how to act and determine their own life. nominal and ordinal properties of a scanned product. Our Page and Kray’s research also highlights that privacy and display is intuitive to use and requires no training. the extent to which a participant benefits are important Participants find it easier to select items when they have issues for determining the ethical acceptability of desirable properties than to not select them because they persuasive technologies. All three of these ethical have undesirable properties. The Augmented Shopping considerations (autonomy, privacy and benefits) are Trolley makes non-obvious nutritional, ethical and shaping the development of the Augmented Shopping environmental product information salient to shoppers and Trolley. facilitates the fast and frugal decision making typically used in a supermarket. Some of the global consequences of To ensure shopper’s autonomy, they will be free to decide selecting particular products can now be made salient to whether they use the Augmented Shopping Trolley and also shoppers at the point of decision making, potentially able to choose which particular non-obvious product facilitating changes in consumer behaviour. We argue that information they want to be informed about. Given that our system is an ethical persuasive technology as it users can configure the system to provide different product enhances the ability of shoppers to buy choose products in information, privacy is not compromised, even though the accordance with their individual values. handlebar will be visible to other shoppers, as they will not understand what particular LED patterns mean. Some of the REFERENCES product information that will be provided by the 1. Cabinet Office and Institute for Government (2010) Augmented Shopping Trolley can clearly benefit a MINDSPACE. Influencing Behaviour through Public participant, for example, nutritional data, whereas other Policy. London: Cabinet Office. information, such as food miles, may not have direct http://www.instituteforgovernment.org.uk/content/133/ personal benefits. In fact, trying to minimize food miles mindspace-influencing-behaviour-through-public-policy may lead, literally, to a personal cost. However, we assume 2. EDS IDG Shopping Report 2007: Shopping Choices: that if participants choose to be informed about a particular Attraction or Distraction? type of product information then they do so because it is of http://www.eds.com/industries/cir/downloads/EDSIDG Report_aw_final.pdf benefit to them and in keeping with their lifestyle choices. 3. Page, R. E. and Kray, C. Ethics and Persuasive We are currently considering how to use the display to Technology: An Exploratory Study in the Context of provide aggregate information about the contents of a Healthy Living. Proceedings of the First International participant’s trolley. The display could indicate how Workshop on Nudge and Influence in Mobile Devices, averaged values of all the participant’s purchases relate to pp. 19-22. some norm(s), for example, is the weekly shop below or 4. Reischach, F., Michahelles, F., Guinard, D., Adelmann, above the average shopper’s food miles. Clearly, there are R., Fleisch, E., Schmidt, A.: An Evaluation of Product normalization issues to be resolved to enable such Identification Techniques for Mobile Phones. In: comparisons to be made. One ethical consideration with Proceedings of the 12th IFIP TC 13 international Conference on Human-Computer Interaction, pp. 804-- this type of display is that even if an observer did not know 816 (2009) what aspect of product information the aggregate display 5. Santilli, P. The Informative and Persuasive Functions of encoded, under certain conditions it could be evident Advertising: A Moral Appraisal. Journal of Business whether a participant was above or below a norm, thereby Ethics, 27--33, 1983. compromising a shopper’s privacy. For example, if the 6. Simon, H. A.: Invariants of Human Behavior. Annual observer had also used the display themselves and the Review of Psychology, 41, 1--19 (1990) colour encoding was fixed. One way to ensure privacy is to 7. Todd, P.M.: How Much Information Do We Need? allow participants to customize aspects of the display, such European Journal of Operational Research, 177, 1317-- as the colour encoding used. A second ethical concern with 1332 (2007) this sort of display is that norms, like salience, typically 8. Todd, P.M., Gigerenzer, G.: Environments That Make Us Smart: Ecological Rationality. Current Directions in influence people unconsciously. To ensure that the Psychological Science, 16(3), 167--171 (2007) autonomy of participants is not compromised it seems 9. Todd, P. M., Rogers, Y. and Payne, S. J. Nudging the important to inform them about the methods used in a Cart in the Supermarket: How much is Enough display and how these typically influence behaviour before Information for Shoppers. In: Proceedings of they choose to use the Augmented Shopping Trolley NIMD2010, pp. 23 – 26 (2010) 10. Underhill, P. Why We Buy: The Science of Shopping. CONCLUSIONS Simon and Schuster: New York. 2009 Our lab-based study shows that participants can rapidly read a shopping trolley handlebar display to determine both