The growing spread of smart objects is changing the way humans interact with technologies since the interaction they propose is more and more physical and less virtual. From an HCI perspective, one of the most interesting aspects regards how non-technical end users can program the behavior of such smart objects. This poster presents an ongoing project on three novel interaction paradigms that support the creation of smart visit experiences.
where the Internet is connected to the physical world via ubiquitous sensors1.
In domains like Cultural Heritage (CH), smart objects
can be installed in museums, archaeological parks and
exhibitions to create smart visit experiences, i.e.
scenario where visitors acquire CH content by
interacting with the surrounding environment and
smart objects included in it. However, to achieve such
engaging scenarios, some issues still need to be solved.
From an HCI perspective one challenging goal regards
how non-technical users (e.g. museum curators), can
be enabled to make multiple smart objects interact
among them. In CH the role of domain experts is
currently quite limited: they can at last configure single
objects that visitors bring across the CH site to receive
personalized content when they reach some interactive
hot spots [
To better understand the meaning and benefits of a smart visit experience, in the following we report on a scenario. The main person is Tyrion, a guide of an archeological park. He typically organizes its tours by arranging visitors in groups of ten persons and going with them in the park, stopping at some of the most significant points to explain their history. Tyrion wants to enrich this format by playing a serious game during the visit. The game he envisioned is based on two types of smart objects: 1) a smart version of the “Mouth of Truth” (MoT), i.e., a Roman marble disc with a relief carving of a man's face whose mouth, according to legend, closes if a liar sticks his hand in it (a smart, small scale version is visible in Figure 1); 2) two decks of twenty smart cards, each depicting an archaeological find. When the tour starts, visitors are assigned to two teams that receive a deck of card. During the visit, every time Tyrion stops at a point of interest, during his narration he poses a question and asks each team to select one card they consider relevant to that question. Visitors have to put the selected card inside the MoT, whose eyes blink green if the card is right and blink red otherwise. In addition, points are assigned/removed to each team depending on their answers. At the end of the game, the winning team wins a park souvenir. Toward Advanced Interaction Paradigms to Program Smart Visit Experiences To create smart visit experiences like the one described in the Tyrion scenario, end users need to program the behavior of smart objects by defining ECA rules. For example, to determine when a smart card is correctly read by the MoT, a rule like the following has to be created by using EFESTO-5W: IF (read smart card code is = AB123456 AND MoT GPS position is lat=12.3456;long=12.3456) THEN the eyes of MoT blink of color (0,255,0) The syntax for rule definition is far the end-user language since it refers to technical aspects, like the smart card hexadecimal code or the the GPS coordinates reached by the MOT during the visit. A more natural rule with the same behavior would be: IF (read smart card is = Egyptian vase AND MoT position is FURNACE) THEN the eyes of MoT blink green This last rule refers to the meaning the expert gives to each smart object, in particular the smart card is associated with the name “Egyptian” instead of and hexadecimal code, while the MoT position is called “FORNACE” instead of specifying GPS coordinates. If experts can define this type of rules, representing an abstraction that speaks their language, they are facilitated in creating a smart visit experience. The current version of the EFESTO-5W prototype lacks the possibility to define sensible attributes to be included in ECA rules. Therefore, this project aims to define novel, intuitive interaction paradigms that permit to extend the native properties of a smart object (events and actions) with custom attributes, in order to exploit them while synchronizing smart objects. The attribute definition becomes a preliminary phase that end users have to carry out before synchronizing smart objects. Given the nature of smart objects, i.e., devices with which users can physically interact through their sensors and actuators, we think that the mechanisms to define such attributes have to take into account the physicality of the interaction. In other words, traditional visual interfaces could not be adequate to define custom attributes, while more concrete and tangible paradigms can better fit the smart-objects world. The starting point of this research was the identification of a set of attributes end users can define on smart objects. We found that data types like string, number and geographic position can be used to define a wide variety of significant attributes on smart objects. Starting from these attributes, we designed three different composition paradigms during a design workshop study involving 28 users arranged in groups of 5/6 participants.
The first paradigm is a tangible solution based on the use of real objects representing the three types of attributes. Initially, each group was asked to identify, for each type of attribute, at least one object of the real world whose affordance refers to the attribute meaning. For example, for the string attribute, objects like pens, inkwell and sheet were proposed. Then, they were asked to propose interaction mechanisms to combine physical attributes with smart objects. All suggestions were mediated by a mobile device. In the resulting paradigm users have to scan (e.g., by using the device camera) the smart object and then the physical attributes representing the type of attribute they want to assign, or vice versa. Each time a physical attribute is scanned, a pop-up appears on the end-user mobile device asking to define the attribute details by writing its name and value.
The second paradigm is a pervasive solution based on the use of the real world as source of attributes. The surrounding environment is conceived as a set of passive objects with their attributes. Let’s think, for example, of a museum in which there are paintings annotated with QR-codes that can be scan with a mobile phone to read detailed information (e.g., style, painter, history). These passive objects could be exploited to copy their attributes and paste them in smart objects. During our study, each group was asked to propose, at a high level, a solution to capture the attributes of passive objects and to send them to the smart objects. In the resulting idea, a smartphone is used to explore the surrounding environment in an augmented reality fashion, but everything is visualized in black and white, except the passive objects that exposes attributes. Indeed, they are augmented with colored pins, each one associated with a type of attribute (e.g. a brown pin for a string attribute). When user approaches a passive object, its pins are enriched by their names and values, according to a semanticzoom technique. Users can collect all the useful attributes, also editing their names/values, and then they can scan a smart object to paste the collected attributes.
The third paradigm is a tactile solution based on the
use of a tabletop interactive display. Conversely to the
previous paradigms, this solution was designed
exploiting their knowledge and expertise on interactive
displays [
This paper has presented an ongoing research aiming at investigating interaction paradigms that support domain experts in the creation of smart visit experiences. Three different composition paradigms have been identified, i.e. tangible, pervasive and tactile paradigms. As future work, we will develop three prototypes, each one implementing a paradigm, to be compared to assess their usability and their support to the creative design of smart interactive experiences.