=Paper=
{{Paper
|id=Vol-1602/paper1
|storemode=property
|title=A Visual Paradigm for Defining Task Automation
|pdfUrl=https://ceur-ws.org/Vol-1602/paper1.pdf
|volume=Vol-1602
|authors=Carmelo Ardito,Giuseppe Desolda,Maristella Matera
|dblpUrl=https://dblp.org/rec/conf/avi/ArditoDM16
}}
==A Visual Paradigm for Defining Task Automation==
https://ceur-ws.org/Vol-1602/paper1.pdf
1
A Visual Paradigm for Defining Task
Automation
Carmelo Ardito Abstract
Università di Bari, In the last years, researchers are devoting many efforts to
via Orabona, 4 improve technological aspects of the Internet of Things
70125 Bari, Italy (IoT), while little attention has dedicated to social and
carmelo.ardito@uniba.it practical sides. Professional developers program the
behavior of smart objects. In addition, often the
Giuseppe Desolda functionality exposed by a single object are not able, alone,
Università di Bari to exhaustively support the end users’ tasks. The
via Orabona, 4 opportunities offered by IoT can be amplified if new high-
70125 Bari, Italy level abstractions and interaction paradigms enable also
giuseppe.desolda@uniba.it non-technical users to compose the behavior of multiple
objects. To fulfill this goal, we present a model to express
Maristella Matera rules for smart object composition, which includes new
DEI – Politecnico di Milanoo operators for defining rules coupling multiple events and
Piazza Leonardo da Vinci, 32 conditions exposed by smart objects, and for defining
20134 – Milano, Italy temporal and spatial constraints on rule activation. Such
matera@elet.polimi.it model has been implemented in a Web application whose
composition paradigm has been designed during an
elicitation study with 25 participants.
Author Keywords
End-User Development of Mashups, Visual Paradigms for
ECA Rule Expression, Internet of Things.
General Terms
Design, Human Factors.
Introduction
The current integrated technologies confer intelligence to
Copyright is held by the author/owner(s). any type of objects (also called things in this document) and
AVI, June 07–10, 2016, Bari, Italy. connect them to the network. For example, a pressure
sensor can make an office door handle smart, for example
alerting a user located in another part of the world every
2
time someone comes into that office. The added value is IoT Challenges
that the object, being part of a network of smart things (the From the technological point of view, such a large number of
Internet of Things (IoT) can communicate with other objects things requires an adequate network infrastructure and
or services, thus triggering dynamic behavior. If the door efficient communication protocols, especially due to the fact
sensor is able to communicate with a sensor worn by a that integrated devices have very limited resources (e.g.,
person, than a remote user can also know who entered the CPU, RAM, memory, battery). Further issues, such as
office. privacy (e.g., in the smart door scenario, if and when it is
allowed to know who entered into the office) and safety
One of the opportunities of the IoT is especially the
(e.g., if a hacker is able to break a device, then it could
possibility to collect in real time information concerning
access all the devices it is connected with) must be
events and behaviors happening in the real world. For
addressed.
example, a potential advantage of the IoT is to anticipate
the needs of a human even before he is aware of it. From the HCI point of view, a major challenge is to enable
even non-technical users to manipulate data and
Today these scenarios may appear unrealistic since we are
functionality of things in a simple and natural way. Today, in
not pervaded with a significant number of sensors. However,
fact, this is a prerogative almost always reserved to
Cisco’s forecasting estimates that more than 50 billion smart
developers who, through the use of specific programming
objects will be deployed by 20201. There are already many
languages, provide pre-packaged solutions to users. The
contexts in which the IoT is adopted. For example, for
most important challenge is to allow non-technical users to
creating wearable devices, i.e. clothing and accessories such
define and manage the connections between things, which
as bracelets, watches, t-shirts, rings, shoes, can integrate
represent the real benefit of IoT, especially in the next years
sensors capable of detecting physiological parameters and
when a growing amount of things will be available.
actuators to communicate with the users. Domotics is
another very active IoT context: pressure, volumetric and Some works in the literature propose mashup techniques for
distance sensors, as well as infrared cameras, are able to addressing this issue. For example in [1] and [3] the
ensure the security of homes; they permit a remote user to authors introduce two systems for the mashup of things for
interact with the devices installed in the house (e.g., lights, home automation, both consisting of two design
heating, blinds) and assist user to avoid hazards such as environments: one is devoted to electrical engineers who
floods, fires or explosions. Other sectors that are define the behavior of devices through a visual
beneficiating of the advent of IoT are Smart Cities, industry representation of logic operations and algebraic formulas;
and environment through energy saving. the second one allows non-technical users to create a web
page where they can include widgets to display data coming
from things and synchronize their behavior based on a
“wired” composition paradigm. The problem is however that
several studies have demonstrated that this kind of
1
http://newsroom.cisco.com/feature- composition paradigm is not suitable for non-expert users
content?type=webcontent&articleId=1208342
[5, 6, 8] as it forces them to deal with concepts like data
3
flow and parameter passing which cannot be mastered by In fact, most of these platforms currently used in the IoT
people who are not expert in programming. One of the domain do not permit (for example IFTTT) or make it
products available in the market is gluethings difficult (for example gluethings) to specify multiple events
(http://www.gluethings.com), a Web platform for registering and actions. Similar difficulties arise when specifying
and composing things. Unfortunately, also this system offers temporal or spatial constraints, e.g., to define the following
a wire-based composition paradigm and also require users behavior: “If I’m in Rome and I post an image on Instagram
to handle JSON to set parameters for the low-level behavior between 8.00 and 11.00 a.m., post the same picture also on
of things – such practices are out of reach for laypeople, i.e., Twitter and Tumblr”.
those users which represent the actual business opportunity.
In order to identify composition paradigms able to guide
To determine the success of the IoT, it is necessary to users in the definition of articulated rules, we wanted to
investigate new approaches that, thanks to high-level elicit the end-user mental model, which is an aspect scarcely
abstractions, can enable non-expert users to compose data explored in the field of task automation, as pointed out by
and functionality of things, as well as the communication [7]. As the seed of our investigation, we were inspired by
among them, by means of “natural” composition paradigm. the 5W model, which is adopted in several domains, such as
journalism and customer analysis, and more in general in
Outlook on the composition paradigm
problem solving, to analyze the complete story about a fact.
As mentioned above, things can be treated as services,
It suggests describing a fact by answering the following
because sensors and actuators have an URI that identify
questions:
them on the Internet. In the case of service mashups,
platforms implementing an event-driven approach, as the Who did it?
one described in [4], permit to synchronize Web services so What happened?
that the event produced by/on a service (e.g., selection of a
When did it take place?
word in a text) triggers an action of another service (e.g., a
search using the selected word). We believe that an event- Where did it take place?
driven paradigm is suitable also for the manipulation of Why did it happen?
things offering the advantage of enabling the composition of
We adopted the 5W model in an elicitation study with 25
things among themselves and also with other Web services.
participants aimed at identifying, with the help of users, a
Similar event-driven platforms (e.g., IFTT -
notation for the specification of task automation rules. The
https://ifttt.com/) are now emerging also for task
customization of the 5W model, which we called Rule_5W,
automation. They make it easy to connect two services
helped us to highlight the elements that are essential for
(things or APIs), choose a trigger, and thus create an action.
creating complete meaningful rules for smart object
They look very promising thanks to the simple and effective
composition. In the Rule_5W model, “Who” is replaced by
composition paradigm, even if it is intended for expert users
“Which” for specifying the services involved in a rule. “What”
and therefore requires programming skills.
indicates the triggered events, as well as the actions to be
4
11
2 3
Figure 1. EFESTO: the interface for rule creation.
activated. “When” and “Where” refer to the specification of, area in which a rule is defined. The left side is for specifying
respectively, temporal and spatial conditions for triggering the triggering events, and the right side is to define the
events and performing actions. Finally, “Why” is used for actions to be activated by the selected services.
reporting a short description to explain the rule behavior to
A wizard procedure, activated by the green “+” button
a human reader, e.g., other users with whom the rule is
highlighted by circle 2 in Errore. L'origine riferimento
possibly shared. The paradigm has then been implemented
non è stata trovata., guides the users in defining the
in a Web platform that extends the capability of EFESTO, a
events in a full-automated fashion. The wizard sequentially
platform for the End-User Development (EUD) of Web
shows some pop-up windows in which the service, the
mashups through which data provided by Web APIs can be
events and the conditions are specified. According to
integrated into unified visualizations [2].
WYSIWYG approach, the wizard steps allow the user to
EFESTO and its 5Ws Composition Paradigm define an event in terms of Which is the service to be
In this section we illustrate through an example the main monitored for detecting the triggering event, What service
features of the 5Ws composition paradigm. A user, who we event has to be monitored, When and Where the event has
suppose is a female, creates a rule to automatically turn on to occur. The specification of When and Where conditions is
the coffee machine and roll-up the shutters when her smart optional. At the end of the wizard procedure, the event is
bracelet detects that she has just woken up or the smart defined and its summary appears under the “Events” area.
alarm clock rings. To create this rule, the user clicks the
Actions can be defined by clicking on the green “+” button
“New Rule” button in the navigation bar (Errore. L'origine
highlighted (circle 3 in Errore. L'origine riferimento non
riferimento non è stata trovata., circle 1) and the
è stata trovata.). The button activates a wizard that helps
“Creating Rule” interface appears. The UI shows the main
the user define an action in terms of Which
5
Figure 2. EFESTO: example of rule including two events and two actions.
service will execute the action as a consequence of the the execution of the rule action(s) if all conditions of all
event(s), What action the service has to perform and When events are satisfied. The “Or” button determines the
and Where the action can be performed. definition of a new event that will cause the execution of the
rule action(s), if the conditions of at least one event are
In EFESTO, users may either define first all the events and
satisfied. Once the rule is created (see Figure 2), it can be
then the actions, or define first a basic rule with one event
saved by entering a short description of the rule (the Why in
and one action and then include new events and new
the 5W model).
actions. Events and actions can be added or removed at any
time fostering a dynamic modification of the running Conclusion
mashup. Further events can be added by clicking one of the One of the cornerstones of the future of the IoT will be to
two green “+” buttons labeled And / Or. Choosing the “And” put in the hands of the end users simple software tools
button starts the definition of a new event that will cause capable of making natural and powerful composition
6
between things. It is unimaginable that this possibility is [7] Ur, B., McManus, E., Ho, M. P. Y. and Littman, M. L. Practical
reserved for a few experts. As already happened in the past trigger-action programming in the smart home. Proc. CHI
with software tools (forums, social networks, CMS) that '14. ACM (2014), 803-812.
made users to evolve from simple consumer to prosumer, [8] Wajid, U., Namoun, A. and Mehandjiev, N. Alternative
even for the IoT have to be designed tools suitable even for Representations for End User Composition of Service-Based
non-expert users. This paper has illustrated how a generic Systems. In End-User Development - ISEUD '11. LNCS 6654.
platform for service mashups can be specialized for the Springer, 53-66, 2011.
composition of services that enable accessing/controlling
smart things. We are currently working on implementing the
needed extensions. A demo, illustrating some preliminary
results, will be given during the workshop.
References
[1] Blackstock, M. and Lea, R. IoT mashups with the WoTKit.
Proc. IOT '12. IEEE (2012), 159-166.
[2] Desolda, G., Ardito, C. and Matera, M. EFESTO: A platform
for the End-User Development of Interactive Workspaces for
Data Exploration. In Rapid Mashup Development Tools -
Rapid Mashup Challenge in ICWE 2015. 591. Springer
Verlag, 63 - 81, 2015.
[3] Guinard, D., Trifa, V., Mattern, F. and Wilde, E. From the
Internet of Things to the Web of Things: Resource-oriented
Architecture and Best Practices. In Architecting the Internet
of Things. 1. Springer Berlin Heidelberg, 97-129, 2011.
[4] Matera, M., Picozzi, M., Pini, M. and Tonazzo, M. PEUDOM: A
Mashup Platform for the End User Development of Common
Information Spaces. In Web Engineering - ICWE '13. LNCS
7977. Springer, 494-497, 2013.
[5] Namoun, A., Nestler, T. and De Angeli, A. Conceptual and
Usability Issues in the Composable Web of Software
Services. In Current Trends in Web Engineering - ICWE '10.
LNCS 6385. Springer, 396-407, 2010.
[6] Payne, S. J. Users’ Mental Models: The Very Ideas. In HCI
Models, Theories, and Frameworks - Toward a
Multidisciplinary Science. Morgan Kaufmann, 135-156, 2003.