"The wheel" - an innovative visual model for interacting with a social web of things G. Biamino, P. Grillo, I. Lombardi, F. R. Simeoni, A. Marcengo, A. Rapp Vernero Telecom Italia R&T Dip. di Informatica - Università di Torino Via Reiss Romoli 274 - 10148 Torino - Italy Corso Svizzera 185 - 10149 Torino -Italy {rossana.simeoni,alessandro.marcengo} {biamino, @telecomitalia.it, grillo,lombardi,vernerof}@di.unito.it amon.rapp@guest.telecomitalia.it ABSTRACT (a) creating and managing links with one another and with the Thanks to the evolution of identification and empowering technolo- users they interacted with, i.e. the ability to maintain a "So- gies, interconnected and intelligent things are becoming popular cial Web" of things and people; in many domains. Depending on the kind of service smart things can offer, the possibility to identify and exchange services with (b) playing an active role in managing the links in the social them arises the significant problem of how users should interact network by establishing friendship relations with people and and communicate with things themselves. It is therefore important other things, based on their knowledge and on the users’ be- to provide a proper interaction model, in order to help users in es- havior. tablishing a correct dialogue with empowered objects of the real (c) structuring, aggregating and synthesizing contents and shar- world. Nevertheless, very little attention has been dedicated to the ing knowledge about themselves and the world around them; conceptualization of a good human-smart object interaction model and of an innovative visualization technique for networks of smart (d) personalizing their interaction with users. connected objects. In this paper we therefore present The Wheel, an interaction con- cept developed for WantEat suit of applications, designed to sup- The idea is that smart things tell us about the world around them, port the visualization and the exploration of identifiable things of the place in which they exist and its history and traditions. This the real world and their connections with other things. We propose world is made of relationships which involve people and other things a paradigm that enables a personalized, social and serendipitous and which evolve over time, given the interaction of people with interaction with networked things, allowing a continuos transition things and other people. A peculiarity of WantEat is that rather than between real and digital world. having a social network of users that discuss and express opinions on things, we propose a social network of people and things, where the two categories intermingle. Keywords Mixed Social networks of people and things, ludic interaction. In such a scenario, a need for innovative forms of interaction allow- ing people to interact with things, to be introduced to their social Categories and Subject Descriptors networks and navigate these networks emerges. Users should be H.5.2 [Information Interfaces and Presentation]: Miscellaneous able to have a sensory contact with things of the real world and, at the same time, enhance their experience by accessing further infor- 1. INTRODUCTION mation which is only available in digital form, e.g., by interacting Empowered things of every day life have become more and more with an avatar of the thing itself. Users should also be able to access important thanks to the spreading of Ambient Intelligence and Ubiq- the world around the thing, which is a complex one made of social uitous Computing. Such a scenario opens new challenges for the relationships with other things and people and of knowledge and creation of innovative forms of communication and interaction be- information about them. Thus, things should act as gateways be- tween people and things of the real world. In the WantEat project tween the physical and the digital realms, welcoming users to their [9] we introduced the idea that interacting with a social web of social network. User interaction with things and their world should smart things is an interesting way of getting in touch with the cul- be as easy and natural and possible, giving users the impression of tural heritage of a territory. We call “smart” things which are em- a real experience, and allowing a smooth transition between real bedded social intelligence, i.e., which are capable of: and digital. Moreover, interaction should be personalized accord- ing to individual user profiles and to the context of interaction (po- sition, time, etc.) in order to support and guide users in their explo- ration of mixed people and thing social networks. At the same time, curiosity-driven exploration should be stimulated, so that users can enjoy an entertaining and serendipitous experience. Workshop on Visual Interfaces to the Social and Semantic Web In order to achieve these goals, we devised the “Wheel”, an inno- (VISSW2011), vative interaction model which allows to get in touch with things Co-located with ACM IUI 2011 Feb 13, 2011, Palo Alto, US. of the real world, communicate with them and visualize and ex- Copyright is held by the author/owner(s). plore their social networks. We therefore exploited novel forms of information visualization and interaction technologies to design an communicating with things and ii) visualizing things and exploring innovative human-object-interaction model. their social network. The second aspect represents the core of our concept, while the first one allows users to identify, communicate The paper is organized as follows. Section 2 illustrates a use case. with and use objects of the real world as a gateway for a serendipi- Section 3 introduces both the concept development and the phys- tous exploration of their networks. ical design. Section 4 presents WantEat, the suite of applications using the here proposed interaction paradigm. Section 5 presents 3.1 Requirements the results of the user interface evaluation. Section 6 concludes the Our main goal is to provide an innovative visual interface for visu- paper. alizing and navigating mixed object and user networks. In order to properly design a valuable interaction model we started 2. USE CASE by identifying a few requirements to allow human-object interac- Our domain of application is gastronomy as a way of getting in tion, mixed people and thing networks exploration and learning. touch with the gastronomic traditions and cultural heritage of a ter- Since we follow the user-centered design approach, our attention ritory. Things to get in touch with include food products, market first focuses on users experience and needs. According to our view stalls, restaurants, shops, recipes but also geographic places and ac- users must be able to i) interact with objects in mobility, here and tors such as cooks, producers, shop owners, etc. Interacting with a now, ii) use common devices, such as their mobile phones, for ob- food product (e.g., a bottle of wine) is a way of getting in touch ject identification, iii) learn something about objects with an easy with the cultural heritage behind that wine, made of stories and interaction modality and iv) communicate with objects, performing traditions, and with its social network, made of grapes, producers, social actions such as tagging, rating or leaving a comment. More- shops but also people who talked about it, people who tasted and over users need to be able to perform several tasks in a transparent liked it, recipes that are perfect to taste the wine with, other prod- and playful way thus improving their learning by doing. According ucts that are "friends" of the wine (e.g., a cheese traditionally served to this aspect other important requirements have to do with the edu- with the wine). As an additional advantage, productive chains can cational aspects related to the application: v) interaction should be also be shortened, allowing users a more immediate contact with playful and entertaining; vi) interaction should support serendip- food producers and their territory. ity and discovery; vii) interaction should stimulate curiosity and A scenario: John at the market. John is a tourist at the farmer’s support learning. Finally, since we wanted to keep interaction as market of Springville and he is interested in the local traditions and natural as possible, we identified three technological requirements: gastronomy. When he gets close to a cheese market stall he notices viii) direct contact with the real world must be guaranteed, ix) no a goat cheese which looks tasty and decides to try it. The cheese infrastructure for object identification is needed (e.g., no embed- tastes delicious and John decides to learn more about it. Using ded electronics or tags are used), x) environment infrastructuring WantEat application for his smartphone, John detects and identi- should be minimized, xi) natural contact and interactivity are ex- fies that goat cheese making it the focus of his exploration. Thus, pected. he learns some basic information about it, discovering that it is pre- pared according to a very ancient recipe by less than ten cheese pro- 3.2 Getting in touch and communicate ducers, all living in Springville. John decides to tag and bookmark In accordance with the previously listed requirements, we devel- this cheese, and then he starts exploring its social network. Things oped the following ways of creating the contact between a user and the system thinks he might be interested in and people with a user a thing (fig. 1(a)): profile similar to his are presented at first. For example, John dis- covers that this cheese goes well together with local rhododendron honey and is served as part of many typical dishes at Springville • Taking a picture. The user frames the label of a product with tavern. His attention is captured by the recipe of a goat cheese the camera; the label is recognized and a contact between the salad and decides to learn more about it. By making it the focus user and the thing is created (fig. 1(b)). of his exploration, he discovers that it is prepared with another lo- cal product, cranberry mustard. Focusing on cranberry mustard, he • Geopositioning. The user can start the navigation by getting discovers that it is sold in a small shop nearby and decides to go in touch with a place (e.g., Springville village in the scenario and buy it. Learning that John needs cranberry mustard in order above) and thus with the things related to the place, i.e., the to prepare a goat cheese salad, the shopkeeper suggests him to buy things around him. a bottle of Tandalay, a local white wine, which, she assures, is a • "Surprise"/"Recommend". The user can get in touch with perfect accompaniment for goat cheese. Once at home, John pre- a random thing or get a recommendation based on his/her pares the salad and tastes it with Tandalay wine. John is amazed: preferences, position and social network. a perfect match! He therefore decides to access WantEat system again and post a comment for the goat cheese, suggesting to try it • Bookmarking. The user can create a list of "preferred" things together with Tandalay wine. Since also other users had suggested and start the navigation from one of them. In other words, the the same match, the goat cheese and Tandalay wine become aware user can have a list of friends including users and things and of each other and a new link is added between them. Now, Wanteat can get in touch with one of these friends at any time. users can discover Tandalay wine by exploring the goat cheese so- cial network, and viceversa. • Searching things. 3. THE WHEEL: AN INTERACTION MODEL In future work we will also consider the possibility that a thing In the this section we describe how we developed the interaction calls the user. If the application is turned on and a user gets close model according to the previously mentioned goals. We first illus- to a thing that the recommender considers relevant for him/her, the trate the requirement analysis. We then discuss in detail our design system can suggest him/her to get in touch with the thing. Since this concept which is made of two main aspects: i) getting in touch and kind of proactive behavior could lead to user distraction, a good balance should be found between information awareness and the need to avoid frequent interruptions. Moreover, direct actions over objects are allowed for leaving or getting information about them as soon as they’ve been detected. Actions available are: rate, tag, comment, add to favorites. Users are able to directly dialogue with the object identified without hav- ing to access the whole system. The user interface (see fig. 1(g)) proposes a simple solution that emphasizes the immediacy of inter- action itself. The object detected is kept in the background and the functionalities are displayed as buttons over it. Thus, users have the impression of acting directly upon objects, just by using their de- (a) (b) vice. The interaction is kept natural and the merge between virtual and real is preserved. 3.3 Interacting with the thing and its world: The Wheel Concept. As stated before, things should act as gateways for ac- cessing the cultural heritage of a territory. Thus, as soon as users have established a contact with a thing, they can both interact with it, and access its social network. We devised an interaction model which supports users in explor- ing such a social network, maintaining a strong focus on the thing which served as a starting point, and at the same time allowing users to discover and interact with its contacts. From a conceptual (c) (d) point of view, the thing the user has got in touch with represents the center of the world the user is going to explore. If the user expresses interest in learning more about this thing, it should be able to tell the user about itself, providing both general knowledge and information synthesized from the interaction with other people (including tags, comments, votes). Moreover, the user should be al- lowed to perform direct actions (see Section 3.2), if she/he wishes to enrich this thing with her/his personal experience. The thing in focus has many different relationships with people and other things in the gastronomy domain: this aspect is represented in our model by visually surrounding it with the members of its social network. These can be grouped in homogeneous subsets (for ex- ample, things related to the territory should be distinguished from things related to cooking) and assigned to different areas around the (e) (f) (g) the central thing, in order to ease user exploration and to provide an effective overview of the available information. Moreover, users Figure 1: Example of the wheel on the iPhone. should have the chance to investigate the relationships which link the central thing and its contacts, discovering: i) more information about the relationship itself (e.g., if two valleys are connected to each other because certain products are produced in both of them, ("People") contains people that are friends of the thing in focus users should know what these products are), 2) if other things exist (e.g., who bookmarked it); the sector "Prodotti" ("Products") con- which are in the same relationship with the central thing. tains other food products that are friends of the thing in focus (e.g., a wine that goes well with a cheese); the sector "Cucina" ("Cui- Design. We developed a "wheel" model. In a wheel, the object sine") contains entities related to cuisine, such as cooks, restau- that occupies the center represents the current focus, while objects rants, recipes. arranged along its circumference clearly appear to be related to it. Moreover, the wheel radiuses embody the relations among objects. Each sector can be expanded by touching it as shown in fig. 1(e). Much research has been devoted to radial menus dating back to The items can be explored by rotating the wheel, as an old style the seminal work of Shneiderman [1], based on the idea that they telephone dialing selector. One item at a time is highlighted and the can significantly decrease selection times. In our model a simple relation between it and the thing at the center is explained. Items form of spatial categorization is provided by partitioning the area in a sector can be ordered in a personalized way, for example ac- surrounding the center in four sections corresponding to four broad cording to user preferences. Notice, however, that items are not macro-categories, which can vary according to the type of thing in further categorized, so that users have to run through a sector in or- focus. Each thing belongs to one of the four sectors. In the exam- der to find something interesting: in this way, our model supports ple of fig. 1(c), the thing in focus is a food product; the first wheel curiosity-driven exploration (rather than goal-oriented search) and ("Territorio" – "Territory") contains the friends related to the ter- promotes serendipity. The user can continue exploration by chang- ritory, the production and supply chain of the thing in focus (e.g., ing the thing in focus. This can be done by simply dragging one producers, shops, production places, etc.). The sector "Persone" thing towards the small wheel in one of the corners (fig. 1(f)). At this point the whole wheel is recomputed and displayed to the user. • User-to-user links. Links between users are either created Using the wheel allows the users to explore a territory via the social explicitly by the users (e.g., links between a user and his/her network of the intelligent things they meet and to exchange infor- friends) or based on the analysis of users’ behavior. The lat- mation with them, thus matching our project requirements. When ter case includes, e.g., the links between users with similar they are interested in an object, users simply have to touch it in profiles or being in the same place at a given time or being order to access the available information. If their interest in this fond of a specific thing or a class of things. object is even stronger, and they want to make it the starting point of their exploration, they just have to drag it to the center of the • User-to-thing links are based on the user’s actions, i.e., a interface. This object becomes the new focus and the surround- user is linked to the things he/she bookmarked, commented, ing related objects are updated accordingly. In addition, notice that tagged, rated positively or visited. users always interact with specific instances (although possibly at • Thing-to-thing links. These links are created by the system different levels of abstraction, as in the case of “Barbaresco” and based on the analysis of the users’ behavior and interaction “red wine”), as it normally happens in real life, rather than with with things. An item X is linked to another item Y in case, for categories in some sort of taxonomy, as it usually happens in GUI- example, X has been mentioned as a tag or in a comment on style menus, thus making interaction more “natural” and decreasing Y, or in case X and Y appear frequently together in the book- their cognitive workload (see requirements). marks of several users, or in case X and Y have been visited frequently on the same occasion by several users. These links 4. WANTEAT are created by tracking and analyzing the actions performed WantEat is a suite of applications we designed that exploits the by the users and looking for significant correlations between interaction model described in this paper. In particular WantEat things. The same approach can be used to create further links Mobile is an application for Apple iPhone while WantEat Backshop between users and things whenever the name of a user is fre- is a web application specifically meant for food producers. quently associated by other users to a thing (or vice-versa). 4.1 Notes on system design Finally WantEat is an adaptive application which maintains an ex- WantEat has been designed as a partially distributed one where we plicit user model and exploits a recommender for selecting and distinguish: (i) a “server side" which is in charge of the creation ranking information to be provided to a specific used based on and management of the network of objects and people and which his/her model. contains the logic for creating intelligent adaptive behavior; (ii) the “client side”, available on mobile smartphones (currently we are One last word about infrastructuring. Minimizing all infrastruc- using Apple iPhone) and on the Web, which is in charge of manag- tures is an important goal of the project. The recognition of the ing the interaction between people and objects and networks; (iii) label of a thing is thus the main way to create a contact with the the infrastructure for interacting with objects. user and this does not require intervention on the things. The server side is constituted by a number of agents. First of all, the system exploits an ontology of the application domain, defin- 5. EVALUATION WITH USERS ing all the concepts in the domain and the relations among them. We adopted a user-centerd approach in the design of the interaction For example, it includes a taxonomy of enogastronomical products model. Thus we performed user tests starting at the early stages of (e.g., of different types of cheeses based on different properties like the project. The first prototype of the wheel has been tested with 12 the type of milk and the production techniques), linking them to users who have been asked to perform a number of tasks, ranging actors (such as producers or market stall owners) and to places in a from thing recognition to navigation with the wheel. The prototype geographic ontology. Inferences on the ontology allow the system was tested also to evaluate the following dimensions: easiness of (i) to associate inferred properties to objects; these pieces of infor- use, clarity, efficiency, effectiveness, usefulness. We also investi- mation can be told by the objects to interested users; (ii) to create gated the level of learnability of the interface. All the users liked links between objects. In fact, one way to create a link between two the service; no major usability problem were pointed out; the users objects is deducing it from some property they share in the domain evidenced some minor issues (e.g. the information about the prod- ontology. ucts should have a different distribution in order to increase the ho- mogeneity in the various sections of the application; the interaction A second important agent is the social network manager. It is in affordances, i.e. "back", "home", etc., should be more clear and charge of managing links in the mixed social networks of things evident; etc.) that have been fixed in the final prototype which is and people. Two ways of managing links are considered. presented in this paper. Although the users found some interaction problems at the first impact with the interface, the level of learn- ability was very high. The final judgments expressed by the users • Links inferred from the ontology, e.g., linking products and about the tested dimensions were positive. The users also provided places or linking products with similar properties. interesting comments about the advantages of using the application • links deriving from the behavior of users. while visiting an area or while visiting a market for discovering products. Finally, the users suggest to improve the service adding information about the prices of the products and the localization on User can interact with things in different ways. On the one hand a map of the places in witch the products are produced or sold. they can add tags, comments and ratings to a thing; on the one hand, things tell stories to users, including information about them as in- In order to further evaluate our concept we exploited WantEat ap- ferred from the ontology and syntheses of the tags and comments plications. In September 2010 we populated Wanteat with informa- provided by the users. This behavior is the basis from dynamically tion about the "Provincia di Torino" (Turin region) and its enogas- creating (and removing) links between things and pople: tronomy. A test of the system has been performed afterwards with two main goals: checking the stability and efficiency of the server guaranteeing a faster, easier, and entertaining access to the world and validating the consistency of data entry. In October 2010 the and networks of things they find interesting. system was presented and tested by a wide public at Salone del Gusto1 . During 5 days of Salone del Gusto the application has 8. ACKNOWLEDGEMENTS been used by several hundreds of casual users, who were very en- The project has been partially supported by Regione Piemonte, thusiastic about the services and opportunities offered by Wanteat. grant ICT Converging Technologies 2007. The authors are grateful The producers also had the opportunity to study the feedback from to Adriano Design (www.adrianodesign.it) for their support in the the users. We collected 684 structured interviews from users who graphic design of the interface. evaluated the system and we will analyze the huge amount of data from this test in the next months and we hope to have some first results for the time of the workshop. 9. REFERENCES [1] J. Callahan, D. Hopkins, M. Weiser, and B. Shneiderman. An empirical comparison of pie vs. linear menus. In Proceedings 6. RELATED WORK of the SIGCHI conference on Human factors in computing In this section we present some of the research literature and ap- systems, CHI ’88, pages 95–100, New York, NY, USA, 1988. plications related to our work. As regards mobile applications con- ACM. nected to the notion of Internet of Things, an interesting project [2] Digg. Digg labs. http://labs.digg.com/. is aPriori [8]. It is an application that aims to connect people to [3] H. Ishii. Tangible bits: beyond pixels. In Proc. of 2nd Intern. products exploiting the power of social recommendations and ob- Conf. on Tangible and Embedded Interaction, pages 15–25. ject identification. With this application users can scan or search ACM, 2008. for products using their mobile phone and they can read, post and [4] A. Kumpf. Trackmate: Large-Scale Accessibility of Tangible share comments about specific objects. This work does not offer a User Interfaces. PhD thesis, Program in Media Arts and real navigation among linked products, and according to our frame- Sciences, School of Architecture and Planning, MIT, June work, it stops at the second interaction modality (direct actions). 2009. [5] P. Liuha, A. Lappetelainen, and J. Soininen. Combining RFID Interesting examples comes from tangible interfaces (TUIs) [3]. with Ontologies to Create Smart Objects. ARTEMIS Tangible computing is a central concept that refers to the augmen- Magazine, 5(9):2633–2654, 2009. tation of the everyday world with computational power, in order to allow entities to manifest natural behavior corresponding to their [6] TouchGraph. Touch graph. environment and to users activities. While this gives people the http://www.touchgraph.com/TGGoogleBrowser.html. opportunity to interact directly through physical artifacts, in our [7] Tweetflare. Tweetflare. http://n96.org/. project we also want to investigate a new way of interaction with [8] F. von Reischach, F. Michahelles, D. Guinard, R. Adelmann, smart things, which beyond their physical behavior, have to natu- E. Fleisch, and A. Schmidt. An evaluation of product rally manifest enhanced behavior. An example of this paradigm is identification techniques for mobile phones. In 12th IFIP Trackmate ([4]) is an open-source tangible tracking system which TC13 Conference in Human-Computer Interaction allows to recognize objects tagged with circular barcodes and to get (Interact’09), 2009. information on their position, rotation and color when placed on a [9] The Piemonte Team, WantEat: interacting with social surface. networks of intelligent things and people in the world of enogastronomy. Proceedings of IUI Workshop on Interacting Our work has similarities with SOFIA, a project that exploits Se- with Smart Objects, Palo Alto, USA February 13, 2011. mantic Web and ontologies in order to represent the objects’ knowl- edge for creating context aware smart objects [5]. As far as network visualization is concerned, some examples are provided by visual search engines [6], social network applications such as Twitter [7] and Digg [2]. 7. CONCLUSIONS We introduced the idea of interacting with a social web of smart things and we discussed an interaction paradigm for supporting this idea. A prototype in the field of gastronomy has been implemented and tested in a real world trial. In this paper, we focused on the novel interaction paradigm (wheel model) we proposed for network visualization and exploration. Such a model allows users to learn information about things they got in touch with, discover related things and people in thier social net- works, learn more about the relationships linking network meme- bers and enrich all these elements with their personal experience. We believe that our interaction model can significantly improve the way people experience the cultural heritage of their territory, by 1 Salone del Gusto (www.salonedelgusto.it) is an event organized by Slowfood in Torino every two years, with about 200000 visitors.