In this paper we present a reward-based field evaluation of the interaction model developed for Wanteat, designed to support the visualization and the exploration of identifiable objects of the real world and their connections with other objects. The interaction model proposes a paradigm that enables a personalized, social and serendipitous interaction with networked things, allowing a continuous transition between real and digital world. We will illustrate the procedure and the results of such evaluation, carried out with a prototype application without an active users community. In particular, we would like to discover if the interaction model stimulates the exploration of the objects in the system and their networks, and if it promotes the interactive features of the application, in particular social actions. Human computer interaction (HCI) ➝ HCI design and evaluation methods
WantEat [2] is an intelligent mobile application in food domain, which puts together real and virtual words. By means of WantEat it is possible to make everyday objects smart and able to communicate with users and to create social relationships with users and other objects. Objects are gastronomic items, such as food products, market stalls, restaurants, shops, recipes but also geographic places and actors such as cooks, producers, shop owners, etc. WantEat is based on the idea that socially smart objects could play the role of gateways for enhancing the interaction between people and a territory and its cultural heritage. If objects could speak they could tell people about the world around them, the place in which they stay and its history and traditions. This world is made of relationships which involve Copyright © 2016 for this paper by its authors. Copying permitted for private and academic purposes. people and other objects and which evolve along time, given the social activities of the objects.
In order to achieve these goals, we devised an intelligent interaction model that enables a personalized, social and serendipitous interaction with networked things, allowing a continuous transition between real and digital world. We therefore exploited novel forms of information visualization and interaction technologies to design an innovative human-object-interaction model.
We decided to conduct a field trial, which is used to evaluate applications in a context of use that is quite close to that of the real life. Kjeldskov et al. [2], evaluating MobileWARD, a contextaware mobile system in a field study, highlighted the limitations caused by the lack of control, as there were no predefined tasks to drive users’ behaviors. Rogers et al. [5] conducted an evaluation in the field of LillyPad, an ubicomp application for learning, pointing that the process was costly in terms of time and effort. Since WantEat, at the time of the evaluation, was still in a prototypical stage, we needed a way for creating a meaningful context for users interacting with such a multifaceted application in a natural way. We needed also a way to encourage users to perform a set of actions and to interact with other users via the social features of the system, even in absence of an active users community. Thus, we decided to use a field trial, conducted in a fair, and enhanced by rewards, to push users to use all the features of our app and to provide the social context for using the app’s social features. For more description about the reward-based methodology of evaluation see [4].
WantEat [2] is a smartphone application that introduces a novel paradigm for supporting the user interaction with social networks of smart objects. This interaction is made of two main phases: (i) getting in touch and (ii) sharing information with the object and exploring its social network.
Getting in touch. A basic assumption of our approach is that infrastructuring should be minimized. We aimed at supporting interaction with everyday objects, with no embedded electronics or tags. Thus, we developed a number of ways of creating the contact between a user and an object (Fig. 1): (i) Taking a picture of the label of a product with the camera (Fig. 1(b)); (ii) Geopositioning the user in a specific place and thus with the objects related to the place, i.e., the objects around him; (iii) Getting a recommendation; (iv) Searching or (v) Exploring bookmarks. information to the object in focus and (ii) influencing the social relations between objects, as discussed in the following. Interacting with the object and its world: The wheel. Once a contact with an object has been established, the user can interact with it and access its social network. Since we aimed at using objects as gateways for accessing the cultural heritage of a territory, we designed an intelligent interaction model that allows users to explore the world starting from a contacted object. We developed a “wheel” model (Fig. 1(c)), where the wheel can be seen as the square of a village, the traditional place for meeting; in this place the user can interact with the object and its friends, exchanging information and knowledge, being introduced to and exploring their social networks. The object the user is interacting with is in the center of the wheel. The user can get in touch with it by simply touching it, which is an appealing and natural way of performing selective actions with a touch sensitive interface. The selected object tells the user about itself, providing both general knowledge and information synthesized from the interaction with other people (including tags, comments, ratings) (Figure 1(d)). The user can, in turn, tell something to the object: in particular, she can add her tags, comments and ratings or can bookmark the object (Figure 1(g)). These actions contribute to (i) adding the When an object is in focus (Figure 1(c)), the wheel provides access to the social network of its friends (both people and objects). Each friend belongs to one of four sectors; the partition into sectors depends on the object in the center. In the example in Figure 1(c), the object in focus is a cheese; the first sector “Territorio” (Territory) contains the friends related to the territory, the production and supply chain (e.g. producers, shops, production places, etc.). The sector “Persone” (People) contains people that are friends of the object in focus (e.g. people who bookmarked it or who wrote a comment on it); the sector “Prodotti” (Products) contains other food products that are friends of the object in focus (e.g. a wine that goes well with a cheese); the sector “Cucina” (Cuisine) contains entities related to cuisine, such as restaurants, recipes, and so on.
Each sector can be expanded by touching it. The expanded sector fills the screen and the items in the sector are displayed as small circles in a ring (see Figure 1(e), where the “Territorio” sector is expanded), similar to the dialer in an old style telephone. The order of the items is based on the user model and on item type (maintaining items of different types and preferring those more suitable for the user). The items can be explored by rotating the ring, in the same way as dialing on the old style telephone. One item at a time is enlarged and the relation it has with the object in focus is highlighted in a small box. See again Figure 1(e), which shows that the object in the center of the wheel (miniaturized in the bottom right corner of the screen) is produced in (“prodotto in”) the place (“Valle di Lanzo”, i.e. Lanzo valley) enlarged in the sector. Information about the enlarged item can be displayed by touching it. The user can continue the exploration by changing the object in focus. This can be done by simply dragging the enlarged item toward the wheel miniature in one of the corners (Figure 1(f)). At this point the whole wheel is recomputed and displayed to the user. For more details on system architecture and the involved software components see [2].
A good occasion to test WantEat interaction model and its features, has been the international food fair Cheese. This biennial exhibition, held in the streets of the town of Bra in Piedmont (Italy) attracts about 300,000 visitors. The objective of this round of test was to stimulate the spontaneous usage of the application in all its aspects and in a context as close as possible to real use. Thus during the four days of the fair, the application has been installed directly on the users iPhones. Participants could use it whatever they would like and for as long as they desired (even for the whole four days duration of the event). The high degree of freedom given to users has been balanced by means of “game missions”, which substituted the formal tasks of the laboratory evaluation. Each participant received a leaflet with instructions and a map of the fair highlighting the areas in which the application was working. Inside the exhibition 10 items (cheeses), were selected as the focus of the evaluation. They were located in different areas of the fair, and were marked as recognizable by the application. The main objective given to users was to recognize at least 5 cheeses by taking a picture taste them and perform some social actions on the application. Users received a prize when they returned to the installation point: a T-shirt with the Application logo. A live leaderboard at the installation point maintained all participants informed of their current score. This reward-based field evaluation (extensively described in [4]) was aimed at evaluating: 1) The interaction model, and in particular if it stimulated the exploration of objects and their network, and if it promoted the serendipitous discovery of new recommended items; 2) The interactive features of the application, and in particular how users communicate with objects by taking advantage of the social features of the application, even in absence of an active users community.
In the four-days event 157 people attended the trial and installed the system on their smartphones. 110 users out of 157 (70.06%) have actively participated in the evaluation. In total they have interacted with 102 objects. In particular, 72 users out of 110 (65.45%) interacted with more than the required 5 objects (AVG=28.08 objects per user, STD=28.08), and around 51.8% of users (namely 57) interacted with more than the 10 selected cheeses (AVG=28.08 objects per user, STD=28.08). As the evaluation instructions directly required interacting with at least 5 objects out of 10 selected in the fair, this result means that the interaction model quite well supported the free exploration of the application contents. Moreover excluding the 10 cheeses that have to be photographed, 51.8% of users browsed the other 92 objects contained in the app (other cheeses, producers linked to the cheeses, etc.). This is a first index that the wheel model supported quite well the serendipitous discovery of new contents present in the application, helping users to follow their own path on the application, for more details see [4].
We classified 70 users (63.64%) out of 110 as contributing users, since they made some sort of social actions (rating, commenting, sharing bookmarks, tagging) that bring contributions to the application contents (for details see [4]). This shows a positive result in terms of social engagement in the context of the rewardbased evaluation, although there was not an already active user community before the beginning of the evaluation. As an example of the results regarding the social features rating and bookmarking actions have been very frequent, as less time-consuming than other actions. For analyzing the action sequences the users made while using the application we have exploited TAXOMO [1], a taxonomy-driven modeler, which given a taxonomy of events and a dataset of sequences of these events produces a compact representation of the sequences. The representation adopted in TAXOMO is a Markov model. The states of the model are nodes in the taxonomy, where the last level (leaves) contains the observable symbols. We have organized the collected data in a simple taxonomy, (Figure 2) where CTPRVS means “all the user actions”, which is the taxonomy top level class, while VS is the sub-class that contains browsing actions, namely the observable symbols V and S, which correspond to the actions “display the object and its network”, namely the thing the user is interacting with and its network, and “more info about the object” of the object. CTPR is the sub-class that contains social actions, namely the observable symbols “commenting”, “tagging”, “(adding) preferred items (bookmarks)”, “rating”.
We have calculated the transitional probabilities for each pair of states “x, y” that determine the probability that the next state will be y given that the current state is x. In TAXOMO the sequences are modeled with Markov models whose states correspond to nodes in the provided taxonomy, and each state represents the events in the sub-tree under the corresponding node. In our model the valid states are V, S, C, T, P, R that correspond to display the object and its network (Fig. 1(c)), more info about the objects (Fig. 1(d)), comment, tag, add bookmark, and rate (Fig. 1(g)) and are respectively grouped in VS (browsing actions) and CTPR (social actions). The transition from x to y is calculated as the ratio between the number of times they occur together and the number of time x occurs.
Considering the so grouped states we have collected the following
transition probabilities:
(
VS
CTPR
$
VS
CTPR
$
From the above transition probabilities we can observe that users
always start the interaction with browsing actions (
The main goals of the interaction model were supporting users in exploring the object network, maintaining a strong focus on the object and at the same time allowing users to interact with its network, discovering new objects in a serendipitous way, and finally allowing users to perform direct actions to enrich the object with their personal experience. From the evaluation results users seem to use the model according to our expectations, in particular the interaction promotes the serendipitous discovery of new related items, helping the users to follow their own path. This is demonstrated by the analysis of user actions, and by the sequence analysis. Thus, we can conclude that the interaction model is quite effective in promoting the desired user interaction with the system. The field evaluation study conducted in a fair, and enhanced by rewards in a game context, gave us the possibility to provide users with the social context needed for interact with the app in a situation close with that of the real life. Although the application was in a prototype phase and thus there was not a pre-existent user community, the evaluation context was able to generate a sufficient amount of interactions useful to evaluate the interaction model and the social features of WantEat (see [4] for details). This is especially true if we compare these results with the ones collected during a previous field trial we conducted at Salone del Gusto 2010, the biggest food fair in Italy, where the game mechanics were not present. During this field trial, 675 users out of 684 have actively participated in the evaluation (98.67%), while just 74 contributing users (10.96%) performed 179 (3,84%) social actions out of 4660 total actions. The reward-based evaluation described in this paper with 70 (63.64%) contributing users on 110 active users allowed users to perform 547 social actions (25.63%) on 2134 total actions.