Thanks to the evolution of identification and empowering technologies, interconnected and intelligent things are becoming popular in many domains. Depending on the kind of service smart things can offer, the possibility to identify and exchange services with them arises the significant problem of how users should interact and communicate with things themselves. It is therefore important to provide a proper interaction model, in order to help users in establishing a correct dialogue with empowered objects of the real world. Nevertheless, very little attention has been dedicated to the conceptualization of a good human-smart object interaction model and of an innovative visualization technique for networks of smart connected objects. In this paper we therefore present The Wheel, an interaction concept developed for WantEat suit of applications, designed to support the visualization and the exploration of identifiable things of the real world and their connections with other things. We propose a paradigm that enables a personalized, social and serendipitous interaction with networked things, allowing a continuos transition between real and digital world.
Empowered things of every day life have become more and more
important thanks to the spreading of Ambient Intelligence and
Ubiquitous Computing. Such a scenario opens new challenges for the
creation of innovative forms of communication and interaction
between people and things of the real world. In the WantEat project
[
The idea is that smart things tell us about the world around them, the place in which they exist and its history and traditions. This world is made of relationships which involve people and other things and which evolve over time, given the interaction of people with things and other people. A peculiarity of WantEat is that rather than having a social network of users that discuss and express opinions on things, we propose a social network of people and things, where the two categories intermingle.
In such a scenario, a need for innovative forms of interaction allowing people to interact with things, to be introduced to their social networks and navigate these networks emerges. Users should be able to have a sensory contact with things of the real world and, at the same time, enhance their experience by accessing further information which is only available in digital form, e.g., by interacting with an avatar of the thing itself. Users should also be able to access the world around the thing, which is a complex one made of social relationships with other things and people and of knowledge and information about them. Thus, things should act as gateways between the physical and the digital realms, welcoming users to their social network. User interaction with things and their world should be as easy and natural and possible, giving users the impression of a real experience, and allowing a smooth transition between real and digital. Moreover, interaction should be personalized according to individual user profiles and to the context of interaction (position, time, etc.) in order to support and guide users in their exploration of mixed people and thing social networks. At the same time, curiosity-driven exploration should be stimulated, so that users can enjoy an entertaining and serendipitous experience.
In order to achieve these goals, we devised the “Wheel”, an innovative interaction model which allows to get in touch with things of the real world, communicate with them and visualize and explore their social networks. We therefore exploited novel forms of information visualization and interaction technologies to design an innovative human-object-interaction model.
The paper is organized as follows. Section 2 illustrates a use case. Section 3 introduces both the concept development and the physical design. Section 4 presents WantEat, the suite of applications using the here proposed interaction paradigm. Section 5 presents the results of the user interface evaluation. Section 6 concludes the paper.
Our domain of application is gastronomy as a way of getting in touch with the gastronomic traditions and cultural heritage of a territory. Things to get in touch with include food products, market stalls, restaurants, shops, recipes but also geographic places and actors such as cooks, producers, shop owners, etc. Interacting with a food product (e.g., a bottle of wine) is a way of getting in touch with the cultural heritage behind that wine, made of stories and traditions, and with its social network, made of grapes, producers, shops but also people who talked about it, people who tasted and liked it, recipes that are perfect to taste the wine with, other products that are "friends" of the wine (e.g., a cheese traditionally served with the wine). As an additional advantage, productive chains can also be shortened, allowing users a more immediate contact with food producers and their territory.
A scenario: John at the market. John is a tourist at the farmer’s market of Springville and he is interested in the local traditions and gastronomy. When he gets close to a cheese market stall he notices a goat cheese which looks tasty and decides to try it. The cheese tastes delicious and John decides to learn more about it. Using WantEat application for his smartphone, John detects and identifies that goat cheese making it the focus of his exploration. Thus, he learns some basic information about it, discovering that it is prepared according to a very ancient recipe by less than ten cheese producers, all living in Springville. John decides to tag and bookmark this cheese, and then he starts exploring its social network. Things the system thinks he might be interested in and people with a user profile similar to his are presented at first. For example, John discovers that this cheese goes well together with local rhododendron honey and is served as part of many typical dishes at Springville tavern. His attention is captured by the recipe of a goat cheese salad and decides to learn more about it. By making it the focus of his exploration, he discovers that it is prepared with another local product, cranberry mustard. Focusing on cranberry mustard, he discovers that it is sold in a small shop nearby and decides to go and buy it. Learning that John needs cranberry mustard in order to prepare a goat cheese salad, the shopkeeper suggests him to buy a bottle of Tandalay, a local white wine, which, she assures, is a perfect accompaniment for goat cheese. Once at home, John prepares the salad and tastes it with Tandalay wine. John is amazed: a perfect match! He therefore decides to access WantEat system again and post a comment for the goat cheese, suggesting to try it together with Tandalay wine. Since also other users had suggested the same match, the goat cheese and Tandalay wine become aware of each other and a new link is added between them. Now, Wanteat users can discover Tandalay wine by exploring the goat cheese social network, and viceversa. communicating with things and ii) visualizing things and exploring their social network. The second aspect represents the core of our concept, while the first one allows users to identify, communicate with and use objects of the real world as a gateway for a serendipitous exploration of their networks.
Our main goal is to provide an innovative visual interface for visualizing and navigating mixed object and user networks. In order to properly design a valuable interaction model we started by identifying a few requirements to allow human-object interaction, mixed people and thing networks exploration and learning. Since we follow the user-centered design approach, our attention first focuses on users experience and needs. According to our view users must be able to i) interact with objects in mobility, here and now, ii) use common devices, such as their mobile phones, for object identification, iii) learn something about objects with an easy interaction modality and iv) communicate with objects, performing social actions such as tagging, rating or leaving a comment. Moreover users need to be able to perform several tasks in a transparent and playful way thus improving their learning by doing. According to this aspect other important requirements have to do with the educational aspects related to the application: v) interaction should be playful and entertaining; vi) interaction should support serendipity and discovery; vii) interaction should stimulate curiosity and support learning. Finally, since we wanted to keep interaction as natural as possible, we identified three technological requirements: viii) direct contact with the real world must be guaranteed, ix) no infrastructure for object identification is needed (e.g., no embedded electronics or tags are used), x) environment infrastructuring should be minimized, xi) natural contact and interactivity are expected.
In accordance with the previously listed requirements, we developed the following ways of creating the contact between a user and a thing (fig. 1(a)):
Taking a picture. The user frames the label of a product with the camera; the label is recognized and a contact between the user and the thing is created (fig. 1(b)).
Geopositioning. The user can start the navigation by getting in touch with a place (e.g., Springville village in the scenario above) and thus with the things related to the place, i.e., the things around him. "Surprise"/"Recommend". The user can get in touch with a random thing or get a recommendation based on his/her preferences, position and social network.
Bookmarking. The user can create a list of "preferred" things and start the navigation from one of them. In other words, the user can have a list of friends including users and things and can get in touch with one of these friends at any time. Searching things.
In the this section we describe how we developed the interaction model according to the previously mentioned goals. We first illustrate the requirement analysis. We then discuss in detail our design concept which is made of two main aspects: i) getting in touch and In future work we will also consider the possibility that a thing calls the user. If the application is turned on and a user gets close to a thing that the recommender considers relevant for him/her, the system can suggest him/her to get in touch with the thing. Since this kind of proactive behavior could lead to user distraction, a good balance should be found between information awareness and the need to avoid frequent interruptions.
Moreover, direct actions over objects are allowed for leaving or getting information about them as soon as they’ve been detected. Actions available are: rate, tag, comment, add to favorites. Users are able to directly dialogue with the object identified without having to access the whole system. The user interface (see fig. 1(g)) proposes a simple solution that emphasizes the immediacy of interaction itself. The object detected is kept in the background and the functionalities are displayed as buttons over it. Thus, users have the impression of acting directly upon objects, just by using their device. The interaction is kept natural and the merge between virtual and real is preserved.
Concept. As stated before, things should act as gateways for accessing the cultural heritage of a territory. Thus, as soon as users have established a contact with a thing, they can both interact with it, and access its social network.
We devised an interaction model which supports users in exploring such a social network, maintaining a strong focus on the thing which served as a starting point, and at the same time allowing users to discover and interact with its contacts. From a conceptual point of view, the thing the user has got in touch with represents the center of the world the user is going to explore. If the user expresses interest in learning more about this thing, it should be able to tell the user about itself, providing both general knowledge and information synthesized from the interaction with other people (including tags, comments, votes). Moreover, the user should be allowed to perform direct actions (see Section 3.2), if she/he wishes to enrich this thing with her/his personal experience.
The thing in focus has many different relationships with people and
other things in the gastronomy domain: this aspect is represented in
our model by visually surrounding it with the members of its social
network. These can be grouped in homogeneous subsets (for
example, things related to the territory should be distinguished from
things related to cooking) and assigned to different areas around the
the central thing, in order to ease user exploration and to provide
an effective overview of the available information. Moreover, users
should have the chance to investigate the relationships which link
the central thing and its contacts, discovering: i) more information
about the relationship itself (e.g., if two valleys are connected to
each other because certain products are produced in both of them,
users should know what these products are), 2) if other things exist
which are in the same relationship with the central thing.
Design. We developed a "wheel" model. In a wheel, the object
that occupies the center represents the current focus, while objects
arranged along its circumference clearly appear to be related to it.
Moreover, the wheel radiuses embody the relations among objects.
Much research has been devoted to radial menus dating back to
the seminal work of Shneiderman [
(b) (c)
(d) (e) (f) (g) ("People") contains people that are friends of the thing in focus (e.g., who bookmarked it); the sector "Prodotti" ("Products") contains other food products that are friends of the thing in focus (e.g., a wine that goes well with a cheese); the sector "Cucina" ("Cuisine") contains entities related to cuisine, such as cooks, restaurants, recipes.
Each sector can be expanded by touching it as shown in fig. 1(e). The items can be explored by rotating the wheel, as an old style telephone dialing selector. One item at a time is highlighted and the relation between it and the thing at the center is explained. Items in a sector can be ordered in a personalized way, for example according to user preferences. Notice, however, that items are not further categorized, so that users have to run through a sector in order to find something interesting: in this way, our model supports curiosity-driven exploration (rather than goal-oriented search) and promotes serendipity. The user can continue exploration by changing the thing in focus. This can be done by simply dragging one thing towards the small wheel in one of the corners (fig. 1(f)). At this point the whole wheel is recomputed and displayed to the user. Using the wheel allows the users to explore a territory via the social network of the intelligent things they meet and to exchange information with them, thus matching our project requirements. When they are interested in an object, users simply have to touch it in order to access the available information. If their interest in this object is even stronger, and they want to make it the starting point of their exploration, they just have to drag it to the center of the interface. This object becomes the new focus and the surrounding related objects are updated accordingly. In addition, notice that users always interact with specific instances (although possibly at different levels of abstraction, as in the case of “Barbaresco” and “red wine”), as it normally happens in real life, rather than with categories in some sort of taxonomy, as it usually happens in GUIstyle menus, thus making interaction more “natural” and decreasing their cognitive workload (see requirements).
WantEat is a suite of applications we designed that exploits the interaction model described in this paper. In particular WantEat Mobile is an application for Apple iPhone while WantEat Backshop is a web application specifically meant for food producers.
WantEat has been designed as a partially distributed one where we distinguish: (i) a “server side" which is in charge of the creation and management of the network of objects and people and which contains the logic for creating intelligent adaptive behavior; (ii) the “client side”, available on mobile smartphones (currently we are using Apple iPhone) and on the Web, which is in charge of managing the interaction between people and objects and networks; (iii) the infrastructure for interacting with objects.
The server side is constituted by a number of agents. First of all, the system exploits an ontology of the application domain, defining all the concepts in the domain and the relations among them. For example, it includes a taxonomy of enogastronomical products (e.g., of different types of cheeses based on different properties like the type of milk and the production techniques), linking them to actors (such as producers or market stall owners) and to places in a geographic ontology. Inferences on the ontology allow the system (i) to associate inferred properties to objects; these pieces of information can be told by the objects to interested users; (ii) to create links between objects. In fact, one way to create a link between two objects is deducing it from some property they share in the domain ontology.
A second important agent is the social network manager. It is in charge of managing links in the mixed social networks of things and people. Two ways of managing links are considered.
Links inferred from the ontology, e.g., linking products and places or linking products with similar properties.
links deriving from the behavior of users.
User can interact with things in different ways. On the one hand they can add tags, comments and ratings to a thing; on the one hand, things tell stories to users, including information about them as inferred from the ontology and syntheses of the tags and comments provided by the users. This behavior is the basis from dynamically creating (and removing) links between things and pople: User-to-user links. Links between users are either created explicitly by the users (e.g., links between a user and his/her friends) or based on the analysis of users’ behavior. The latter case includes, e.g., the links between users with similar profiles or being in the same place at a given time or being fond of a specific thing or a class of things.
User-to-thing links are based on the user’s actions, i.e., a user is linked to the things he/she bookmarked, commented, tagged, rated positively or visited.
Thing-to-thing links. These links are created by the system based on the analysis of the users’ behavior and interaction with things. An item X is linked to another item Y in case, for example, X has been mentioned as a tag or in a comment on Y, or in case X and Y appear frequently together in the bookmarks of several users, or in case X and Y have been visited frequently on the same occasion by several users. These links are created by tracking and analyzing the actions performed by the users and looking for significant correlations between things. The same approach can be used to create further links between users and things whenever the name of a user is frequently associated by other users to a thing (or vice-versa). Finally WantEat is an adaptive application which maintains an explicit user model and exploits a recommender for selecting and ranking information to be provided to a specific used based on his/her model.
One last word about infrastructuring. Minimizing all infrastructures is an important goal of the project. The recognition of the label of a thing is thus the main way to create a contact with the user and this does not require intervention on the things.
We adopted a user-centerd approach in the design of the interaction model. Thus we performed user tests starting at the early stages of the project. The first prototype of the wheel has been tested with 12 users who have been asked to perform a number of tasks, ranging from thing recognition to navigation with the wheel. The prototype was tested also to evaluate the following dimensions: easiness of use, clarity, efficiency, effectiveness, usefulness. We also investigated the level of learnability of the interface. All the users liked the service; no major usability problem were pointed out; the users evidenced some minor issues (e.g. the information about the products should have a different distribution in order to increase the homogeneity in the various sections of the application; the interaction affordances, i.e. "back", "home", etc., should be more clear and evident; etc.) that have been fixed in the final prototype which is presented in this paper. Although the users found some interaction problems at the first impact with the interface, the level of learnability was very high. The final judgments expressed by the users about the tested dimensions were positive. The users also provided interesting comments about the advantages of using the application while visiting an area or while visiting a market for discovering products. Finally, the users suggest to improve the service adding information about the prices of the products and the localization on a map of the places in witch the products are produced or sold. In order to further evaluate our concept we exploited WantEat applications. In September 2010 we populated Wanteat with information about the "Provincia di Torino" (Turin region) and its enogastronomy. A test of the system has been performed afterwards with two main goals: checking the stability and efficiency of the server and validating the consistency of data entry. In October 2010 the system was presented and tested by a wide public at Salone del Gusto1. During 5 days of Salone del Gusto the application has been used by several hundreds of casual users, who were very enthusiastic about the services and opportunities offered by Wanteat. The producers also had the opportunity to study the feedback from the users. We collected 684 structured interviews from users who evaluated the system and we will analyze the huge amount of data from this test in the next months and we hope to have some first results for the time of the workshop.
In this section we present some of the research literature and
applications related to our work. As regards mobile applications
connected to the notion of Internet of Things, an interesting project
is aPriori [
Our work has similarities with SOFIA, a project that exploits
Semantic Web and ontologies in order to represent the objects’
knowledge for creating context aware smart objects [
As far as network visualization is concerned, some examples are
provided by visual search engines [
We introduced the idea of interacting with a social web of smart things and we discussed an interaction paradigm for supporting this idea. A prototype in the field of gastronomy has been implemented and tested in a real world trial.
In this paper, we focused on the novel interaction paradigm (wheel model) we proposed for network visualization and exploration. Such a model allows users to learn information about things they got in touch with, discover related things and people in thier social networks, learn more about the relationships linking network memebers and enrich all these elements with their personal experience. We believe that our interaction model can significantly improve the way people experience the cultural heritage of their territory, by 1Salone del Gusto (www.salonedelgusto.it) is an event organized by Slowfood in Torino every two years, with about 200000 visitors. guaranteeing a faster, easier, and entertaining access to the world and networks of things they find interesting.
The project has been partially supported by Regione Piemonte, grant ICT Converging Technologies 2007. The authors are grateful to Adriano Design (www.adrianodesign.it) for their support in the graphic design of the interface.