In this paper, we present a preliminary study aimed at improving the users' mental model of an automatic smart home system based on trigger-action rules. We hypothesized that a computational model of how the rules are evaluated and activated, coupled with a linguistic form of the rules that clarifies the difference between events and states, may improve accuracy in identifying buggy rules, which may eventually increase end-users' acceptance of this type of systems. This pilot study was conducted with non-programmers and provided some evidence in support of this hypothesis.
End-User Development (EUD) is defined as the possibility for people without programming
experience to create or modify their applications [12]. EUD approaches focus on empowering users
beyond their involvement in the design phases, as advocated by user-centered design, and propose a
vision in which design, learning, and development are an inherent part of technology in use [
Programming is difficult for non-experts because it often requires expressing solutions in ways that
are not familiar to them [15]. The concept of rule may provide an intelligible metaphor for programming
digital technologies since it embeds the idea that specific actions need to be taken in specific situations
[
In the context of IoT-based smart devices, the programming approach based on contextual rules has evolved in the so-called trigger-action programming for which a rule takes the specific form of an action that is performed upon the occurrence of condition. Such a metaphor is supposed to be easily graspable by users since IoT devices are usually either sensors that detect events in the world or actuators that operate changes in the world (of both).
The simplicity of this approach is demonstrated by the success of web-based services, like IFTTT
(https://ifttt.com/) [20]. Nevertheless, for an effective programming of IoT devices, it is important to
allow more expressive triggering conditions and more elaborate actions [
Indeed, mental models are important cognitive constructs that can explain how people interact with
the world and specifically with complex artifacts [
Mental models can and should be communicated to the user by proper design [14] but it might also
be effective to communicate them verbally, by providing the user with an adequate description of how
the system works. Regardless of their correctness, different mental models of the same object may lead
to dramatically different user-object interactions: Halasz and Moran’s study [
The present study aims at investigating how end-users conceptualize EUD in smart homes and at providing simple but effective design hints to improve understanding of trigger-action rules.
Specifically, we would like to encourage accurate mental models by providing a better representation of how rule-based systems works and by fostering a recognition of different important concepts by imposing linguistic constraints to the rules.
Regarding the first aspect, there is some of evidence that proper and short instructions may
contribute to develop accurate mental models [11] and that mental models arising from an explicit
description of the working mechanism (hereafter called “computational models”) are more effective
than models arising from a simple description of the procedure/rules of a given system (hereafter called
“descriptive models”) [
Regarding the second aspect, one key point is the difference between events and states [10]. In some
EUD implementations there is the possibility to associate a condition involving events and/or states
with the trigger [
The study is based on the one presented by Brackenbury and colleagues [
Before the study, the users are offered a short tutorial with some examples about the distinction between events and states (see Figure 1) and they have been told that rules have the form: WHEN <event> WHILE <state or set of states> DO <list of actions>. However, they can be shortened as IF <event and/or set of states> THEN <list of actions>.
Following this, the scenarios were presented in random order. In the rules of the scenarios, the use of the WHEN/WHILE/DO form is alternated (in a randomized way) with the form IF/THEN.
Half of the subjects receive a computational description of how the system works that specifies the cyclical nature of the rule evaluation-and-activation system (i.e., a “computational model”) while the other half receive a simpler description of the possible rule structures (i.e., a “descriptive model”).
Therefore, the study has two between-subject conditions (i.e., computational vs. descriptive models; cf., Hypothesis 1) and two within-subject conditions (i.e., the two possible format/structure of the rules; cf., Hypothesis 2).
The study has been piloted with 12 bachelor students from the Department of Psychology and Cognitive Science of the University of Trento; none of them had knowledge of, or prior experience with, computer programming.
Although the results are preliminary, they are promising and clearly show a trend that supports both hypotheses.
0.86 0.84 0.82 )10.80 0 :ge0.78 n (ra0.76 y c rua0.74 c c an0.72 a eM0.70 0.68 0.66 0.64 descriptive
computational Model
WHEN/WHILE IF
The results of this pilot study are initial evidence supporting our hypotheses: Our manipulations seem to be effective in increasing the participants’ understanding of the rules’ behavior and of their effects. Consistent with our hypotheses, the use of a WHEN/WHILE format for the rules and (possibly) the computational description of the system seems to improve users’ mental models of how the smarthome automatic system works.
The full study is ongoing. It will include a measure of comprehension of the training and more detailed analyses of the results.
This work has been supported by the Italian Ministry of Education, University and Research (MIUR) under grant PRIN 2017 "EMPATHY: EMpowering People in deAling with internet of THings ecosYstems" (Progetti di Rilevante Interesse Nazionale – Bando 2017, Grant 2017MX9T7H). [10] Huang, J. and Cakmak, M. 2015. Supporting mental model accuracy in trigger-action programming. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing - UbiComp ’15 (Osaka, Japan, 2015), 215–225. [11] Kulesza T., et al., 2012. Tell Me More?: The Effects of Mental Model Soundness on Personalizing an Intelligent Agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’12). [12] Lieberman, H., Paternò, F., Klann, M., Wulf, V., 2006. End-user development: an emerging paradigm. In: Lieberman, Henry, Paternò, Fabio, Wulf, Volker (Eds.), End User Development.
Springer, The Netherlands, pp. 1–8. [13] Norman, D.A. 1983. Some Observations on Mental Models. In Gentner, D. and Stevens, A.L. eds.
Mental models. Erlbaum. [14] Norman, D.A. 2013. The design of everyday things. Basic Books. [15] Pane, J.F. et al. 2001. Studying the language and structure in non-programmers’ solutions to programming problems. International Journal of Human-Computer Studies. 54, 2 (Feb. 2001), 237–264. [16] Paternò, F. 2013. End User Development: Survey of an Emerging Field for Empowering People.
ISRN Software Engineering. 2013, (2013), 1–11. [17] Paternò, F. and Santoro, C. 2019. End-user development for personalizing applications, things, and robots. International Journal of Human-Computer Studies. 131:120–130. [18] Payne, S.J. 2009, Mental Models in Human-Computer Interaction. In Sears, A. and Jacko, J.A.
eds. Human-computer interaction. Fundamentals. CRC Press. [19] Ur, B. et al. 2014. Practical trigger-action programming in the smart home. Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14 (Toronto, Ontario, Canada, 2014), 803–812. [20] Ur, B. et al. 2016. Trigger-Action Programming in the Wild: An Analysis of 200,000 IFTTT Recipes. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI ’16 (Santa Clara, California, USA, 2016), 3227–3231. [21] Yarosh, S. and Zave, P. 2017. Locked or Not?: Mental Models of IoT Feature Interaction.
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17 (Denver, Colorado, USA, 2017), 2993–2997.