=Paper=
{{Paper
|id=Vol-2082/paper_4
|storemode=property
|title=An End-User Development Approach for Crafting Smart Interactive Experiences
|pdfUrl=https://ceur-ws.org/Vol-2082/paper_4.pdf
|volume=Vol-2082
|authors=Carmelo Ardito,Paolo Buono,Maria Francesca Costabile,Giuseppe Desolda,Rosa Lanzilotti,Maristella Matera,Antonio Piccinno
|dblpUrl=https://dblp.org/rec/conf/chi/ArditoBCDLMP18
}}
==An End-User Development Approach for Crafting Smart Interactive Experiences==
https://ceur-ws.org/Vol-2082/paper_4.pdf
Interactive Experiences SmartObjects '18, in conjunction with CHI '18, Montreal, Canada
An End-User Development Approach for
Crafting Smart Interactive Experiences
Abstract
Carmelo Ardito Giuseppe Desolda Antonio Piccinno
Despite the advantages that Internet of Things (IoT)
University of Bari Aldo Moro University of Bari Aldo Moro University of Bari Aldo Moro
technology offers, there are still important issues to be
via Orabona, 4 via Orabona, 4 via Orabona, 4
solved to increase its practical impact. The
70125 - Bari, Italy 70125 - Bari, Italy 70125 - Bari, Italy
opportunities offered by IoT can be amplified if new
carmelo.ardito@uniba.it giuseppe.desolda@uniba.it antonio.piccinno@uniba.it
approaches, based on high-level abstractions and
adequate interaction paradigms, are conceived to
Paolo Buono Rosa Lanzilotti
involve directly non-technical users in configuring the
University of Bari Aldo Moro University of Bari Aldo Moro
behavior of their smart objects. In this paper, we
via Orabona, 4 via Orabona, 4
present our End-User Development approach, which we
70125 - Bari, Italy 70125 - Bari, Italy
would like to discuss at the workshop together with the
Paolo.buono@uniba.it maria.costabile@uniba.it
challenges our future research implies.
Maria Francesca Costabile Maristella Matera Author Keywords
University of Bari Aldo Moro Politecnico di Milano Internet of Things; Smart Object Modelling; Smart Visit
via Orabona, 4 Piazza Leonardo da Vinci, 32 Experience; Cultural Heritage.
70125 - Bari, Italy 20134 - Milano, Italy
ACM Classification Keywords
maria.costabile@uniba.it Maristella.matera@polimi.it
Software and its engineering~Integrated and
visual development environments • Software
Copyright © 2018 for this paper held by its author(s). Copying and its engineering~Visual languages.
permitted for private and academic purposes.
Introduction and Motivation
Smart objects are increasingly pervading the
environments we live in. If enabled to exploit the
abundance of resources (object functionality, produced
data, related applications), end users could compose
the “behavior” of the surrounding environment to
accommodate their everyday needs. However,
programming the behavior of smart objects is currently
23
Interactive Experiences SmartObjects '18, in conjunction with CHI '18, Montreal, Canada
a prerogative reserved for professional developers, as it these disadvantages occur because the proposed
requires the use of scripting languages and tools that platforms are too “general”, claiming that one single
can also vary depending on the underlying hardware. design might satisfy the requirements of many
Furthermore, the available objects often expose very domains. For example, in Cultural Heritage (CH),
specific functionalities that do not result in useful guides and curators are non-professional in Computer
services able to accommodate users’ needs. Science who might create objects that visitors of CH
sites can bring with themselves, touch and manipulate
In order to directly involve non-technical users in
for experiencing the site by receiving personalized
configuring the behavior of their smart objects, new
information. However, by observing CH experts
approaches, based on high-level abstractions and
engaged in the definition of smart objects behavior, we
adequate interaction paradigms, have to be conceived.
realized that, even without being required to write any
Our research capitalizes on years of experience on End-
code, it is still difficult for them to manage technical
User Development (EUD), a discipline that
concepts. We, therefore, propose higher-level
encompasses methods, techniques, tools, and
abstractions to allow end users themselves (i.e., CH
methodologies to allow professionals of a given domain
experts in the case study proposed in this paper) to
to master domains in which they are not professionals
define custom properties to characterize the semantics
[1-4]. In the IoT context, we developed EFESTO-5W, a
of smart objects, thus helping them to make sense of
platform implementing an EUD approach to customize
the available smart devices and digital resources and
and synchronize the behavior of resources, like smart
facilitate the definition of their cross interactions. These
objects and Web services, through Event-Condition-
features have been implemented in EFESTO-SE, an
Actions (ECA) rules [5]. With respect to other Task-
extension of EFESTO-5W, which was evaluated
Automation Systems (TASs) (see, for example, those in
involving 14 professional cultural-heritage guides who
[6]), whose adopted graphical notations often do not
were asked to create an interactive visit experience.
match the mental model of most users [7], EFESTO-5W
The study showed that our hypotheses on extending
promotes a richer set of high-level abstractions and
the semantics of smart objects by means of custom
operators to define rules and a visual notation that,
attributes were valid with respect to the expectations of
despite the intrinsic complexity related to managing
CH stakeholders. We are further exploring and verifying
events and actions, is affordable even by non-
this aspect in some recently undertaken research
programmers.
activities. In particular, we are interested in evaluating
TASs are typically conceived as general purpose if our proposal can stimulate the creativity of CH
systems, but their generality often implies a scarce stakeholder as smart-experience designers: the
adoption by specific communities of end users [8]. hypothesis is that if the smart objects make evident the
EFESTO-5W is also general purpose but can be relationship they have with the content, CH
customized to several application domains. Our professionals can better identify how to adopt such
position, which also derives from observing people devices to convey the CH-site content to visitors.
adopting our EUD tools during field studies, is that
24
Interactive Experiences SmartObjects '18, in conjunction with CHI '18, Montreal, Canada
This paper is organized as follows. The next section A usage scenario
describes, with the help of a usage scenario, the Molly is the CH guide of the archaeological park of
challenges of defining a smart interactive experience Egnathia, an ancient Roman city in Southern Italy.
After the outdoor visit through the ruins, where Molly
and how our approach addresses them. The last section
explains the history of the city and illustrates the
concludes the paper and suggests some research issues function of every place in the park, the visit continues
that would matter to further discuss at the workshop. in the park museum. Molly engages visitors in playing a
serious game in the “smart” rooms of the museum.
Smart Experiences in the CH domain
Here, display cases containing ancient objects are
Even in the CH domain, IoT research has primarily instrumented with sensors able to detect NFC coins1
focused on technical features, e.g., how to program provided to each visitor before the game starts. During
networks of sensors and actuators and how to ensure the game, Molly asks different questions and,
their interoperability [9-12]. Very few approaches try to accordingly, sets the sensors of the display cases in
facilitate the configuration of smart objects and their different modalities by means of an app installed on her
smartphone. For example, she sets the “Age” modality
advantage is limited to programming single objects that
and asks visitors to find the display cases where Roman
the visitors bring across the CH site to receive objects are shown. The visitors move through the
personalized content when they reach hot spots [13]. It museum, identify the cases matching Molly’s request
is still hard for CH experts (e.g., site curators and and touch them with their coin. If they are successful,
professional guides) to synchronize the behavior of the light inside cases turns green and the visitor’s
multiple devices in order to create smart visit current score is increased. Then, Molly asks other
questions and sets the display cases in the
experiences where different sensors and actuators,
corresponding modality, thus the game continues. The
installed in the environment or embedded in tangible synchronizations between cases and NFC coins are
objects manipulated by visitors, actively react to some established by the guide using our platform through the
detected events. creation of ECA rules.
Defining the smart visit experience
In our ongoing research, we are investigating how non-
From the previous scenario, it is evident that the
technical users, like CH guides, can personalize the
personalization of a smart visit experience might not be
behavior of smart objects in order to customize visit
limited to a trivial synchronization of smart objects, but
experiences depending on their skills and capabilities,
it might also require creating digital narratives threads
as well as on the needs of specific visitors. To
that professionals themselves need to put in context
understand how ECA rules can be exploited to define
with respect to the CH-site content. Driven by these
smart visit experiences, let us consider the scenario
emerging requirements, we introduce the notion of
described in the following section.
custom attributes, as a means to characterize smart
objects not only by native events and actions (as
conceived in many IoT platforms) but also by properties
1 Near-field communication device, resembling an ancient Roman
coin, used for identifying each visitor.
25
Interactive Experiences SmartObjects '18, in conjunction with CHI '18, Montreal, Canada
that the domain experts (i.e., the designers of the the CH guide needs to define, for each display case,
smart experience) can define to assign semantics to the couplings with the NFC coins, she has to create a rule
objects. Such semantics empowers and simplifies the for each coupling. In addition, such rules refer to
creation of ECA rules, as it can exploit an enriched technical terminology (e.g., the NFC-coin code) that
vocabulary based on user-defined terms. Visual does not correspond to the language adopted by the
mechanisms also simplify the creation of custom domain experts. Figure 1 illustrates an example of ECA
attributes and their association to smart objects. rule that defines the synchronization between a single
case (i.e. case 3) and a specific coin. Thus, Molly has to
To understand some of the advantages of custom
replicate this rule for coupling all the other cases and
attributes, let us go back to the above scenario. Since
coins.
Figure 1. Example of ECA rule determining the behavior of a single case and coin.
26
Interactive Experiences SmartObjects '18, in conjunction with CHI '18, Montreal, Canada
In our proposal, before creating ECA rules, Molly parks in Southern Italy. The guides, who participated
interacts with a visual tool offered by our platform, individually and underwent the same procedure, were
which allows her to assign custom attributes to each required to design a smart visit experience.
case by manipulating widget interfaces, without the
We collected different qualitative data. All the
need of coding. Custom attributes can be seen as
interactions were audio-video recorded by using an
conceptual tools that can allow designers to
external camera. To evaluate user satisfaction, a
characterize the basic elements of a smart experience
questionnaire with 23 statements was administered at
(i.e., smart objects and rules) with a semantics related
the end of the study. The first statement was the Net
to the content to be conveyed during the smart
Promoter Score (NPS) question [14], typically used to
experience.
measure, on a scale between 0 and 10, how likely users
In the example of Figure 2, she defines and assigns the would recommend the product to a friend or a
attributes Age, representing the age of the artifacts colleague. The NPS score is equal to 57, i.e., excellent;
contained by the cases, Points, representing the it indicates an attitude towards suggesting this system
number of points the visitor gains if the answer is to other CH guides. This encouraging result is also
correct, Blinking time, indicating for how many seconds confirmed by the analysis of the SUS questions, which
the case has to blink. From now on, the creation of ECA gave us detailed indications about the perceived system
rules can exploit this terminology (see for example usability and learnability. The SUS global score
Figure 3). In addition, more general rules, i.e., was 81.1/100 (SD = 14.1), which is higher than the
parametric, can be created. In Molly’s scenario, she average SUS scores (69.5) computed based on one
does not need to define a multitude of very similar thousand studies reported in [15].
rules for coupling every single case and coin, since they
The thematic analysis carried out on the transcribed
are all encompassed by the single rule shown in Figure
data, triangulated with the questionnaire results,
3.
allowed us to identify important themes, also
Evaluation with CH Guides highlighting the presence of some usability issues to be
We recruited 14 professional guides (5 female) aged addressed. We summarize them in the following
between 18 and 50 (x̅ = 37.9, SD = 8.2), operating in section, since we are confident that they would enable
different museums, archaeological sites and natural further discussion at the workshop.
27
Interactive Experiences SmartObjects '18, in conjunction with CHI '18, Montreal, Canada
Figure 2. Platform tool for defining and assigning custom attributes.
Figure 3. A single rule determining the behavior of multiple cases and coins.
28
Interactive Experiences SmartObjects '18, in conjunction with CHI '18, Montreal, Canada
Discussion and conclusions than ECA rules creation. Other metaphors for the
This article has presented our perspective on the EUD property assignment, than the one we implemented in
of smart experiences. The work described here provides EFESTO-SE, could be perceived as more usable. For
the first results of a larger research that aims to example, one participant suggested a spreadsheet-
promote smart objects as components of a smart based solution: users could use a tabular format in
experience that can bring with themselves evident which they allocate smart objects in rows, CA names in
connections with the semantics of content for which columns, and then specify CA values in cells located at
they facilitate the access. We showed how we extended the intersection between rows and columns. The
and customized a generic composition paradigm, tabular format was also adopted in the elicitation study
initially conceived for the EUD of IoT systems, to by the CH guides to specify CAs and their values.
respond to the need of exploiting IoT to mediate
Stimulating creativity in smart-experience design.
narrative and content-appropriation goals for
Another important aspect in smart-experience design is
interactive visits to CH sites. However, some challenges
the adoption of paradigms that can stimulate creativity.
are still open. We summarize them in the following with
The evaluation study demonstrated that EFESTO-SE
the aim of stimulating discussion among the workshop
has a potential in supporting the design of smart
participants. Even though the reported aspects
experiences. However, discussions with participants
emerged from the application of our EUD paradigm to
revealed that there is still room for improvements.
the CH domain, we believe they can be of general
validity and should be taken into account when defining Supporting and fostering technical skills growth.
EUD frameworks for IoT also in other domains. Another aspect emerged during the discussion with the
CH experts regards the customization activities that go
Constraining the flow of design activities. Based
beyond the smart-object programming supported by
on the observed problems and participants’
EFESTO-SE. Indeed, 6 out of 14 participants stated
suggestions, we believe that our environment for the
that, after a certain period of EFESTO-SE usage, they
creation of smart experiences should be redesigned to
would like to be supported in extending the smart
provide a robust guidance to users. For example, a
object capabilities by integrating new sensors and
wizard procedure can guide users in configuring an
actuators, avoiding to involve every time IT experts.
initial, limited core set of smart objects, together with
Even if this activity seems an aspect that only
their CAs and basic ECA rules controlling them. Later,
technicians can accomplish, today there are hardware
users can freely continue expanding this core set until
and software solutions that satisfy this goal. For
obtaining the final and complete smart experience.
example, mCookies2 is an alternative to Arduino that
Simplifying the paradigm for CA definition. As can support people who have an interest in the “Do It
emerged from triangulating questionnaire results with Yourself” paradigm for electronics. It consists of a set
users’ comments, CA definition resulted more difficult
2 http://microduinoinc.com/products/mcookie/
29
Interactive Experiences SmartObjects '18, in conjunction with CHI '18, Montreal, Canada
of magnetic, color-coded modules, which can be staked Y. Dittrich, G. Fischer and A. Piccinno (eds.).
in a LEGO fashion. Springer LNCS 6654, 53-66.
8. Fabio Casati. 2011. How End-User Development Will
References
Save Composition Technologies from Their
1. Carmelo Ardito, Paolo Buono, Maria Francesca
Continuing Failures. In End-User Development - IS-
Costabile, Rosa Lanzilotti and Antonio Piccinno.
EUD 2011. M. Costabile, Y. Dittrich, G. Fischer and
2012. End users as co-designers of their own tools
A. Piccinno (eds.). Springer. LNCS 6654, 4-6.
and products. Journal of Visual Languages &
9. A. Chianese and F. Piccialli. 2014. Designing a
Computing, 23, 2: 78-90.
Smart Museum: When Cultural Heritage Joins IoT.
2. Maria Francesca Costabile, Daniela Fogli, Piero
In Proceedings of the International Conference on
Mussio and Antonio Piccinno. 2007. Visual
Next Generation Mobile Apps, Services and
Interactive Systems for End-User Development: A
Technologies (NGMAST '14), 300-306.
Model-Based Design Methodology. IEEE
10. Vincenzo Mighali, Giuseppe Del Fiore, Luigi Patrono,
Transactions on Systems, Man, and Cybernetics -
Luca Mainetti, Stefano Alletto, Giuseppe Serra and
Part A: Systems and Humans, 37, 6: 1029-1046.
Rita Cucchiara. 2015. Innovative IoT-aware Services
3. Gerhard Fischer, Elisa Giaccardi, Yunwen Ye, Alistair
for a Smart Museum. In Proceedings of the
Sutcliffe and Nikolay Mehandjiev. 2004. Meta-
International Conference on World Wide Web (WWW
design: a manifesto for end-user development.
'15 Companion), 547-550.
Communications of the ACM, 47, 9: 33-37.
11. Francesco Piccialli and Angelo Chianese. 2017. The
4. Henry Lieberman, Fabio Paternò, Markus Klann and
Internet of Things Supporting Context-Aware
Volker Wulf. 2006. End-User Development: An
Computing: A Cultural Heritage Case Study. Mob.
Emerging Paradigm. In End User Development. H.
Netw. Appl., 22, 2: 332-343.
Lieberman, F. Paternò and V. Wulf (eds.). Springer
12. Francesco Piccialli and Angelo Chianese. 2017. A
Netherlands, 1-8.
location‐based IoT platform supporting the cultural
5. Giuseppe Desolda, Carmelo Ardito and Maristella
heritage domain. Concurrency and Computation:
Matera. 2017. Empowering End Users to Customize
Practice and Experience, 29, 11.
their Smart Environments: Model, Composition
13. Daniela Petrelli and Monika Lechner. 2014. The
Paradigms, and Domain-Specific Tools. ACM Trans.
meSch project – Material EncounterS with digital
Comput.-Hum. Interact., 24, 2: 1-52.
Cultural Heritage: Reusing existing digital resources
6. Gabriella Lucci and Fabio Paternò. 2015. Analysing
in the creation of novel forms of visitor’s
How Users Prefer to Model Contextual Event-Action
experiences. In Proceedings of the International
Behaviours in Their Smartphones. In End-User
Committee for Documentation of ICOM (CIDOC '14)
Development - IS-EUD 2015. P. Díaz, V. Pipek, C.
14. Douglas B Grisaffe. 2007. Questions about the
Ardito, C. Jensen, I. Aedo and A. Boden (eds.).
ultimate question: conceptual considerations in
Springer International Publishing. LNCS 9083, 186-
evaluating Reichheld's net promoter score (NPS).
191.
Journal of Consumer Satisfaction, Dissatisfaction
7. Usman Wajid, Abdallah Namoun and Nikolay
and Complaining Behavior, 20: 36.
Mehandjiev. 2011. Alternative Representations for
15. Aaron Bangor, Philip Kortum and James Miller.
End User Composition of Service-Based Systems. In
2009. Determining what individual SUS scores
End-User Development - IS-EUD 2011. M. Costabile,
mean: Adding an adjective rating scale. Journal of
usability studies, 4, 3: 114-123.
30