42 End-User Development for Lifelogging and eWellness Stefano Valtolina Abstract Università degli Studi di Milano The growing popularity of Internet of Things' devices, Via Comelico, 39/41 sensors, and applications available on the market is 20135 Milan, Italy pushing towards a revolution of lifelogging and its valtolina@di.unimi.it relationship with wellness. Thanks to the characteristics of pervasiveness and mobility of wearable IoT tools, we Barbara Rita Barricelli are witnessing the emergence of a new research and Università degli Studi di Milano development domain: the eWellness. Together with Via Comelico, 39/41 this, new challenges arise, and ask for effective 20135 Milan, Italy methods and techniques to improve the way extraction, barricelli@di.unimi.it merge, analysis, visualization, and data sharing are currently managed. In this paper, we present our research study on the eWellness domain and in particular the design and development of the SmartFit framework, aimed at supporting the collaborative visual design of IoT rules by means of End-User Development techniques. Copyright is held by the author/owner(s). Paste AVI, the07–10, June appropriate 2016,copyright/license Bari, Italy statement here. ACM now Author Keywords supports three different publication options: End User Development, Internet of Things, Design, • ACM copyright: ACM holds the copyright on the work. This is the historical approach. Human Factors; ETL (extract, transform and load) • License: The author(s) retain copyright, but ACM receives an operations. exclusive publication license. • Open Access: The author(s) wish to pay for the work to be open access. The additional fee must be paid to ACM. ACM Classification Keywords This text field is large enough to hold the appropriate release statement H.3.4 [Systems and Software]: Distributed systems, assuming it is single-spaced in Verdana 7 point font. Please do not change the size of this text box. Information networks; H.4 [Information Systems Each submission will be assigned a unique DOI string to be included here. Applications]: Miscellaneous; H.5.2 [User Interfaces]: GUI, Interaction styles. 43 Introduction contents to respond to their situational needs. In this Nowadays we are witnesses of the proliferations of scenario, end users find themselves at the center of a different sensor devices able to produce several types complex scenario that they need to manage in efficient, of data that can be profitable used for advising people effective, and satisfactory way. EUD represents the on how to improve their lifestyle, health, and wellness. ideal approach for empowering end users and make The survey reported in [1] underlines that many tools them becoming unwitting developers in their own in the Internet of Things (IoT) field, mainly focus on environment [3,4,5]. hardware devices (sensors and actuators) and on A scalable and efficient solutions should be devised that artificial intelligence techniques that may help to add can be applied by domain experts in configuring the sophisticated capabilities to the processing of network of sensors (both physical and social), and information provided by hardware devices. A current services. ETL (extract, transform and load) solutions, challenge in this domain concerns the need to conceive usually applied off-line, need to be revised and applied it as a socio-technical system, which encompasses not on-line for feeding the system with fresh and timely only a variety of hardware and software components, data. Finally, a user-friendly environment has to be such as sensors, actuators, mobile apps and Web conceived for properly supporting the user in the applications, but also people that are bound together analysis processes (sensor discovery, feeding, and by social ties and personal relations and that are knowledge inference). virtually linked each other. This leads to what is to be Specifically, in this paper we present and discuss our considered the Internet of Things ecosystem. In this sociotechnical study on the design and development of context, the main interest is to provide the users with IoT ecosystems to be used by non-professional sport tools able at properly helping them in making sense of teams. The peculiar structure of non-professional sport the data originated from different sources (i.e., devices, organizations is characterized by the existence of small sensors) and for enabling them to understand and teams with athletes who live different kind of lives, exploit huge amount of data and events that are being professionals in different domains and meeting eventually collected in time. In our research, we mainly only for some hours a week. Keeping track of their study applications of IoT technologies designed for a habits, in terms of physical activity, nutrition, sleep and specific scenario: The eWellness domain in which so on, would help the coaches in understanding the people can collect data gathered through several variety of the team members and finding successful devices for monitoring and keeping track of weight, schemes of training. In order to enable domain experts sport/fitness activity, calories intake, and sleep quality. in generating, configuring and managing the network of In this domain, what is currently missing is a solution sensors and services we adopt a meta-design able to support domain users (experts in monitoring or approach. By means of this approach we seek to create in supervising physical activity of individual or teams) social-technical conditions for broad participation in in selecting relevant data sources and compose them design activities at both design and use time of all the through End-User Development (EUD) techniques [2], involved stakeholders, rather than anticipating all so that users are enabled to self-construct their requirements at design time [6]. Objective, techniques 44 and processes are observed to empower users to be Meta-Design Phase: The StreamLoader active designers, creating their own situated solutions At meta-design level, IoT Engineers need to configure that can be translated in a configuration of sensors and the network of sensors and services for managing the services. data-flow to be served at the Design level interactive system. The outcome of this environment is to detect a The SmartFit Case Study set of events that coaches and trainers need to In this section, we present a use case we studied to manipulate at design level for monitoring the physical experiment the potentials of our solutions both at activities or daily behavior of their athletes. meta-design and design level. In this context of use we StreamLoader provides IoT Engineers with a set of ETL consider three different Communities of Practice (CoPs) operators used for manipulating streams of data in real [7] that co-exist in a eWellness scenario: IoT time (or almost real time). In this way, data are loaded engineers, Coaches and Trainers, and Athletes. IoT from a set of sensors and services, providing a sensors/devices engineers are in charge of connecting, convenient way for user to real-timely read the data maintaining, and setting up the devices and sensors to information and make tactical decisions. Commercial be used by coaches, trainers, and athletes. Their systems such as Talend Studio 1, StreamBase Studio 2, system named StremaLoader, allows the design of data Waylay.io 3 offer graphical interfaces for designing flows by aggregating data sources and applying workflows and dataflows as graphs of connected nodes operators to them for merging, joining and representing tasks and data-sources. While Talend transforming the gathered data. More details about this works on static data coming from fixed data-sources, environment are referred in [8,9]. The StreamLoader StreamBase and WayLay can receive and analyze output is a flow of events that can be used by coaches continuous data streams and are specifically designed and trainers for defining rules to monitor particular for IoT. These environments provide rich user interface situations and to adopt suitable actions according to the support for the full application lifecycle but they are occurrence of given conditions. Exploiting a rule desktop-based systems and in some cases, specific language, the Rule Editor enables coaches and trainers conditions can be only created by adopting strategies to act as end user developers by designing the ECA based on programming languages paradigms (as for statements to be used to supervise athletes’ StreamSQL in StreamBase Studio) or by personalizing performances and lifestyle and they also analyze the existing templates having well-defined trigger policy (as gathered data in their interactive system. Finally, in Waylay.io). Visual queries are other examples of athletes can be seen as the real end users, the ones solutions can be used for assisting users to extract who passively use the data gathered by the IoT sensors information from databases in an intuitive, visual and and devices to better their lifestyle and sport performances. A tailored interactive system can be 1 www.talend.com used to have a view on their behavior and 2 www.streambase.com performances at any time during the day. 3 www.waylay.io 45 natural approach, making information systems transform data coming from different sensors. It relies comprehensive and efficient for a wide range of on the idea of providing a visual workflow generator for applications. SmartVortex Visual Query System [10] is letting the domain experts to create combination of a typical application where the visual strategy of data originated by sources. In Figure 1, the IoT querying takes the form of drawings or graph. Also in engineer has taken data coming from an application this case the expected users have limited competence used for annotating the calories intake and three for managing data-streams but they wish to easily sensors: Two for counting steps, and one for express basic queries with little efforts and no monitoring the sleep. The engineer has then used a competence in developing code. Starting from the merge operator for merging data coming from the two study of current solutions based on graphical visual step-counters and a transform operator for strategies for executing ETL operations, we designed a transforming the minutes of sleep in hours. Finally, meta-design environment for helping the IoT engineers s/he has joint these data into a flow that has been then to manipulate events. In our approach, one of the most collected into the data-warehouse. An advanced use of important EUD activity takes place during the meta- such visual paradigm allows the users to have an online design of the flow of events that characterize a context generation of sample data coming from the data of use. For example, in a gym, IoT engineers need to sources dragged-and-dropped on the canvas, or as combine data coming from different sensors and result of the operations carried out on them (see Figure devices for properly defining the exercises. These EUD 1 (B)). Following this strategy, information is gathered activities are exploited for executing efficiently and from the network of devices and specific operations can effectively Event ETL operators. So far, several be triggered on user requests. operations have been developed for processing and combining the streams produced by the sensors [8]. In the fitness example, the events might concern the user’s physical condition, which will be used for suggesting what exercises are better to perform, or which precautions to follow for improving the quality of live (for example, by changing the diet plan). In our context, the meta-design environment is a Web environment where IoT engineers can drag and drop different sensor data sources and visually apply on them a set of operations. This application offers an engine and graphical environment for data transformation and mashup. As depicted in Figure 1 (A), this mash-up consists of a user interface that contextually displays icons of data sources or operations in order to link, aggregate, merge or Figure 1. A screenshot of the StreamLoader 46 Design Phase: Rule Editor sources. The visual strategies adopted by following The meta-design environment described earlier in this IFTTT or Yahoo's Pipes compliant solutions are Section and used by IoT Engineers, constitutes the promising techniques but, in our opinion, they present base for providing coaches and trainers with the some lacks. The former offers a very simple and easy possibility to manipulate the flow of events in order to to learn solution based on the definition of ad hoc rules monitor the physical activities or daily behavior of their that can notify the end users when something happens athletes. The design environment for manipulating – e.g. when their favorite sites are updated, when they these events aims at offering a graphical visual check-in in some places or their friends do, or warn strategy for exploiting the potentials of an IoT them when specific weather conditions are going to environment in the deployment of rules. Systems like take place. However, the language is not enough JBoss Drools 4, OpenRules, and IBM WebSphereJRules 5 expressive for the specification of more sophisticated provide platforms for supporting users in creating rules based on time and space conditions. On the other complex rules to trigger proper actions when specific hand, the latter offers a too complex solution for conditions occur. In these environments, the coding of supporting the end user in expressing their complex rules is generally performed by skilled preferences. Pretending that end users can deal with technicians through ad-hoc Rule Engines. Nevertheless, APIs of several sensors/devices put at risk the success in real contexts of use, the definition of rules is of the visual approach. Moreover, events in each performed by domain experts that are not experienced stream of an IoT scenario, are time and space technician and prefer to use natural language or dependent and so the related rules need to take into graphical notations. Other visual strategies typically account these type of conditions. Nevertheless, in the used in IoT field for modelling ECA (Event-Condition- described systems, time and space dimensions are Action) rules can be described through the most almost neglected. Once the flow of events for a given famous systems that apply them: IFTTT, Atooma and analysis have been identified by IoT engineers, rules Yahoo's Pipes. The first two examples allow users to should be defined for specifying the action to be define sets of desired behaviors in response to specific actuated when specific events occur. Our system, events. The visual strategy aims at creating automated named Rule Editor, extends the IF-THIS-THEN-THAT rules by using graphical notation for programming approach and supports the definition of rules in a more statements such as: "IF this DO That" or "WHEN trigger articulated way by keeping the complexity at an THEN action". A second type of applications stems from acceptable level. The idea is to keep the simplicity of the outstanding work done with Yahoo's Pipes. These the IF-THIS-THEN-THAT system pairing it with the use applications are based on the idea of providing a visual of formula languages. Moreover, time and space pipeline generator for supporting end users in creating dimensions are exploited and adopted for expressing aggregation, filtering, and porting of data originated by more loose rules in the statements. From the point of view of language expressiveness, we consider rules 4 http://www.drools.org/ that are more expressive than the IF-THIS-THEN-THAT 5 http://www.ibm.com/software/websphere/ system from different perspectives. Complex conditions 47 are supported to correlate events both from the temporal and spatial dimensions. The time dimension allows end users to set rules that can be fired at some specific time, delayed in case of certain conditions are verified, and may be repeated until some event happens. The space dimension gives users the chance of linking rules to the place/area where the events Figure 3. The creation of a temporal condition in the Rule currently taking place, where they will possibly be in Editor system. the future or where they are moving into. Moreover, An example of Composite Rule creation is given in rules are enabled depending on spatio-temporal Figure 2, where the user has defined four rules: (i) R1: conditions that are clearly outlined in the rule Hours of sleep less than 7 (ii) R2: Calories intake at specification. This allows the identification of the dinner greater than 1,500 (iii) R3: Number of steps less streams that are really relevant in a given time and than 8,000 (iv) R4: Activity duration less than 45 given location for monitoring given situations and (minutes). The rules are built in this way: R1 AND R2 properly react to them. By exploiting a Rule Language, AND (R3 OR R4). The meaning of the Composite Rule the rules can be specified by means of the graphical created in this example is: “if the hours of sleep of the Rule Editor. As depicted in Figure 2 the graphical day before are less than 7 AND if the calories intake at interface is based on drop-down menus that are dinner (before the sleep) is greater than 1,500 AND (if populated by using the attributes that characterize the the number of steps is less than 8,000 at day OR the JSON of the flow of events produced by the IoT duration of physical activity is not less of 45 minutes at Engineers with the SmartLoader environment. day THEN send the athlete a message that warns about the behavior and performances. The rule editor aims at allowing non-technical people to specify rules by using simple drop-down menus. The conditions can be composed by combining groups of statements connected by using the operator AND or OR. Temporal conditions are defined as depicted in Figure 3 and use the automatically assigned names of the Rules as elements to be composed (R1, R2, R3, ...). An example of temporal condition is that R3 (number of steps less than 8,000) has to take place before R2 (calories intake Figure 2. The composition of the rules of a CompositeRule. at dinner greater than 1,500) with a range of +/- 5 minutes. In other words, the dinner must begin within 5 minutes after the user has finished to walk (for less of 8000 steps). Another complex temporal condition 48 can be: R4 (activity duration less than 45 minutes) 3. Costabile, M. F., Mussio, P., Parasiliti Provenza, L. begins before of the R2 (Calories intake at dinner and Piccinno, A. 2008. End users as unwitting software developers. In Proc. of WEUSE 2008. greater than 1,500) and ends after +/- 10 minutes. In ACM, 6-10 other words, the trainer wants to check if her/his 4. Costabile, M. F., Mussio, P., Parasiliti Provenza, L. athletes eat too much and how, if they eat not seated and Piccinno, A. 2008. Advanced Visual Systems at a table but when they are on the move and quickly Supporting Unwitting EUD. In Proc. of AVI 2008. (in less than 45 minutes). Once a rule is created, it is ACM, 313-316. stored in a repository for further re-use or for sharing it 5. Barricelli, B.R., Marcante, A., Mussio, P., Parasiliti among members of a community of trainers and Provenza, L., Valtolina, S. and Fresta. G.: BANCO: coaches. In this way, it easy for other users to a Web Architecture Supporting Unwitting End-User customize existing rules according to their needs. Development. IxD&A, 5-6, pp. 23--30 (2009). 6. Fischer, G., Giaccardi, E., Ye, Y., Sutcliffe, AG. and Conclusions and Future Works Mehandjiev, N. 2004. Meta-Design: A Manifesto for In this paper, we have presented an approach for End-User Development. CACM 47, 9, 33-37. supporting IoT engineers, coaches and trainers in 7. Wenger, E. 1999. Communities of practice: designing and analyzing data-flows coming from a set Learning, meaning, and identity. Cambridge of physical/social sensors and services. At the moment university press. we are working for setting up a user test evaluation for 8. Mesiti, M., Valtolina, S., Ferrari, L., Dao, M.-S. and our eWellness system thanks to collaboration with the Zettsu, K. 2015. An editable live ETL system for CSI (Centro Sportivo Italiano – Italian Sport Centre). In Ambient Intelligence environments. WF-IoT (2015), 393–394. this use case, a set of sport teams will be involved in order to perform a set of tasks. What we want to study 9. Valtolina, S., Barricelli, B.R. and Mesiti, M. 2015. from the user tests, is how far our approach is able to End-User Centered Events Detection and Management in the Internet of Things. Current offer new possibilities both at the design and use time Trends in Web Engineering. Springer. 77–90. and to understand how the idea to combine the design 10. Bauleo, E., Carnevale, S., Catarci, T., Kimani, S., and end user environments appears to be successful Leva, M. and Mecella, M. 2014. Design, realization and effective solution both for domain and technical and user evaluation of the smartvortex visual query experts. system for accessing data streams in industrial engineering applications, JVLC. 25, 5, 577-601. References 1. Sadri, F. 2011. Ambient intelligence: A survey. ACM Computing Surveys 43, 4, 1-66 2. Lieberman, H., Paternò, F., Klann, M., and Wulf, V. 2006. End-User Development: An Emerging Paradigm. In End-User Development. Springer, 1- 8.