The amount of information available to help decide what foods to buy and eat is increasing rapidly with the advent of concerns about, and data on, health impacts, environmental effects, and economic consequences. But this glut of information can be distracting or overwhelming when presented within the context of a high time-pressure, low involvement activity such as supermarket shopping. How can we nudge people's food shopping behavior in desired directions through targeted delivery of appropriate information? We are investigating whether augmented reality can deliver relevant 'instant information', that can be interpreted and acted upon in situ, enabling people to make more informed choices. The challenge is to balance the need to simplify and streamline the information presented with the need to provide enough information that shoppers can adjust their behavior toward meeting their goals.
Increasingly we are told about the risks, costs, and benefits of particular food choices. In response, a flood of information is becoming available, online, on food labels, in information leaflets and books, from a variety of sources, aimed at informing the consumer so that better decisions can be made while shopping. But all this information risks overwhelming and overloading the shopper trying to navigate the complex store environment in a hurry, leading to the opposite outcome—poor decisions made without the proper input. How can all this information be consolidated, pruned down, and presented to supermarket shoppers in an easy to understand and meaningful form that will actually help them make better choices about values they care Copyright is held by the author/owner(s).
MobileHCI 2010 September 7-10, 2010, Lisboa, Portugal.
ACM 978-1-60558-835-3/10/09.
about? Technology pundits and researchers are beginning to
promote ‘augmented reality’ that uses Smartphones and other
ubiquitous technologies as the latest solution to this problem.
Kuang [
Technology for ubiquitous information delivery must balance giving people enough new information to improve their decisions against overwhelming them with new things to consider. Ambient information displays, as already used in homes and offices to provide feedback about energy consumption and nudge users toward greater conservation, may strike the right balance in food purchase and consumption as well. However, as we discuss below, moving beyond momentary nudges toward long-term behavior change requires providing detailed-enough feedback to enable learning what to do in the future, for instance on the next shopping trip. We argue that we must improve our (currently limited) understanding of whether and how people attend to and learn from visualizations of multi-dimensional information while engaged in an ongoing activity such as food shopping, using cognitive science models of decision-making and learning together with design principles for information visualization and interaction design.
Rational theories of decision-making [e.g., 15] posit that making a
choice involves weighing up the costs and benefits of different
courses of action. When alternatives are ordered on more than one
relative dimension, this involves compensatory strategies where
information is processed exhaustively and trade-offs made
between features. Such strategies are very costly in computational
and informational terms – not least because they require the
decision-maker to find a way to compare apples and oranges.
Non-compensatory strategies may be used instead as a form of
bounded rationality where not all of the available information is
used and trade-offs can be ignored [
At the same time, recent consumer surveys reveal that shoppers
are demanding more information about the products they buy and
are becoming increasingly aware of the global consequences of
the decisions they make [
However, there is a scarcity of research on how people use
multidimensional information under time pressure and the extent to
which it effects rapid decision-making [
But even though these kinds of visualizations are mostly targeted at non-expert users, they are essentially visual query languages that require considerable cognitive effort to interpret. Can relevant dimensions of products such as food be represented in simple ways that can be glanced at and perceived rapidly to guide shopping decisions in situ?
We propose that rather than providing ever more information to enable consumers to compare products in minute detail when making a choice, a better strategy is to design technological interventions that provide just enough information and in the right form to facilitate good choices. One solution is to exploit new forms of augmented reality technology that enable ‘informationfrugal’ decision-making, in the context of an intensive activity replete with distractions (i.e., shopping in a supermarket or deciding at the kitchen table what to have for dinner). An important consideration when representing multiple dimensions that can be glanced at and perceived rapidly is to enable comparisons to be made and cumulative information inferred in situ. For example, simple contrasting icons (e.g., thermometer icons, percentage bars, balls that change in color) can be presented which increase or decrease in amount in relation to the values being represented. Another approach is to fuse relative measures on different dimensions (e.g., greenness, price, fat level) into singular displays where shape carries the salient information, such as a rectangle that gets taller to convey a nutritional dimension that is general (healthiness) or specific (e.g., salt content) and wider to convey price. A third dimension, such as ‘greenness’, could be added by filling in the rectangle with a shade from red to green to show the amount of carbon emissions for that product. Similar to the idea behind Chernoff faces, the visualizations will be placed side by side to enable quick comparisons.
Another important question is whether to use ‘emotive’
visualizations that can persuade people to select food items they
might not otherwise choose. Various persuasive technologies have
recently been developed to encourage people to take more
exercise. Examples include Fish‘n’Steps [
What then is a good way to provide appropriate information
quickly and simply to shoppers in order to aid their
decisionmaking during the hectic, distracting setting of a trip to the
supermarket? Here we assume the shoppers have selected a
particular dimension that they care about and want to change in
terms of their buying behavior—for instance, choosing products
that are lower fat, or more sustainably grown. To inform shoppers
about how they are doing in achieving this particular goal during
their shopping expedition, cumulative values of the dimensions of
interest across all products chosen so far could be summed up and
displayed in an ambient manner as the current ongoing overall
score “projected” onto the handle of the shopping cart as a color.
For example, a green handle could signify that the shopper has
obtained a ‘carbon footprint’ or ‘fat content’ score below their
target (or below some population average), while a red handle
would indicate that the cart’s contents are above the desired level,
with intermediate levels indicated by intermediate colors (see
Figure 1).
Such an ambient and publicly visible display must first be studied
to see if it fits with how people want to shop, or engenders
unexpected side-effects. Will people be more or less likely to
change their behavior when information about the contents of
their shopping cart is publicly visible for all to see rather than
being privately displayed? Would shoppers try to fill their cart
with healthy and green foods and on finding they were under the
average then treat themselves to luxury goods high in fat and food
miles? Would having their shopping cart glow green at the
checkout, indicating the contents were well below the average, make
them feel good in front of other shoppers [
There are at least three approaches that can be taken to solving this problem, which is essentially one of allocating global feedback appropriately to individual choices of products (akin to the “credit assignment” problem in machine learning). First, we could leave it all up to the users, and assume (or hope) that when they end their shop with a “green” cart, they will buy more things like those the next time around, and when they get a “red” cart, they will buy different things next time. This leverages the human shopper’s intelligent ability to learn from diffuse reinforcement over time, but it will probably be slow, requiring many shopping outings before reliable change occurs. Second, to speed up this process, we could provide more specific feedback about each product that goes into the cart, for instance momentarily flashing the ambient display with a color corresponding to the box of sugar-frosted chocolate bombs or bag of figs being chosen. This will allow shoppers to make more targeted decisions about each product, provided they remember that individual feedback.
Third, to remove the need for such memory, a further interface can be developed to let shoppers query how they should adjust their purchases to come closer to their goal. This could take two main forms. A Comparative Tool could run as a ‘private’ mobile application on a smartphone or PDA and be displayed on the device or somewhere in the environment, such as the shopper’s hand or the product package itself. After identifying the product via a photo or code scanner, the tool will show the product values on the dimensions of interest, and indicate whether this product helps or hinders the achievement of the current shopping goal. This interface could also be used in a comparative manner, scanning two or more products while they are still on the shelf and then showing at a glance which product is best based on the selected dimensions.
As a second ‘off-line’ form of providing more explicit feedback, a Collaborative Tool running on a home computer or surface display would allow shoppers to find out further information about the products they have bought once they get them home, along with input from their families. Multiple users could reflect and discuss together the decisions behind their food purchases with a view to attaining their goals at their next weekly shop, exploiting collaborative planning and social pressures that take place in a family setting. An interactive planner application would enable family members to find out more about particular dimensions (e.g., nutritional values) on a product, meal, or weekly-shop basis, and provide recipe-specific visualizations enabling items to be swapped. For example, a suggestion by dad to cook coq-au-vin for dinner will show it is low on ‘greenness’ (because of a large carbon footprint). This is a dimension the son has selected as an informational layer. Alternative items can be swapped with the chicken, such as tofu, which may then be shown by the application to have a higher greenness value (i.e., smaller carbon footprint). Finally, specific shopping lists could be generated that would achieve the goals set by the shopper and others involved.
To test whether any of these approaches succeeds in nudging shoppers’ behaviour in specific directions within a reasonable time-span, both lab-based experiments and field studies are needed. One line of investigation must assess how the different information displays for the tools described above affect user decision-making strategy, focusing on when and how the interactive display of information enables fast and frugal decisions. This must then be tested further in supermarket studies, using techniques such as mobile eye tracking, observation and talk aloud methods to determine what people look at and how they use the comparative and cumulative tools. Longitudinal studies are also needed to determine whether the tools proposed have long-term impact on behavior, and how quickly such change occurs. Various kinds of households (e.g., family, young people, retired single) should be compared in terms of whether and how their shopping patterns and meal planning behavior change when using the tools—different groups of people may be more or less influenced by different types of nudges, and we cannot assume a one-size-fits-all approach.
Whether these various kinds of information delivery can help move people in the direction of better decisions—in the food shopping domain, or in other applications—remains to be seen. Emerging research suggests that simple visualizations can be designed to be information-frugal and emotive – encouraging people to change their behavior at the point of decision-making. But the trick will be balancing frugality and simplicity with enough feedback detail to allow people to change their choices at a pace that is sufficiently rapid and noticeable to be rewarding and motivating for long-term behavior change.
Thanks to Ricky Morris for creating Figure 1.
[2] Consolvo, S., Klasnja, P., McDonald, D. W., et al. (2008) Flowers or a robot army?: encouraging awareness & activity with personal, mobile displays. Proc. UbiComp'08, 54-63.