=Paper=
{{Paper
|id=Vol-1602/paper7
|storemode=property
|title=A Multi-layered Context Modelling Approach for End Users, Expert Users and Programmers
|pdfUrl=https://ceur-ws.org/Vol-1602/paper7.pdf
|volume=Vol-1602
|authors=Sandra Trullemans,Beat Signer
|dblpUrl=https://dblp.org/rec/conf/avi/TrullemansS16
}}
==A Multi-layered Context Modelling Approach for End Users, Expert Users and Programmers==
https://ceur-ws.org/Vol-1602/paper7.pdf
36
A Multi-layered Context Modelling
Approach for End Users, Expert Users
and Programmers
Sandra Trullemans Abstract
WISE Lab Context awareness plays an important role in smart envi-
Vrije Universiteit Brussel ronments and embedded interactions. In order to increase
Pleinlaan 2
user satisfaction and acceptance, context-aware solutions
1050 Brussels, Belgium
should be controllable by end users. Over the last few years
strullem@vub.ac.be
we have therefore witnessed an emerging trend of visual
programming tools for context-aware applications based
on simple “if this then that” rules. Unfortunately, existing
solutions do not support the easy reuse of the “this” part
Beat Signer in other rules. Further, the desired level of control varies
WISE Lab among individuals. In order to let users choose the right
Vrije Universiteit Brussel level of automation and control, we propose a multi-layered
Pleinlaan 2
context modelling approach distinguishing between end
1050 Brussels, Belgium
users, expert users and programmers. We report on our on-
bsigner@vub.ac.be
going development of the Context Modelling Toolkit (CMT)
consisting of the necessary context modelling concepts as
well as a rule-based context processing engine. We further
discuss an initial design of the graphical user interface for
the presented multi-layered context modelling approach.
Author Keywords
Context-aware systems; context modelling toolkit; end user.
Copyright is held by the author/owner(s).
AVI, June 07–10, 2016, Bari, Italy ACM Classification Keywords
H.5.m [Information interfaces and presentation (e.g., HCI)]:
Miscellaneous
37
Introduction ontology-based approaches are not ideal for dynamic con-
It is well-accepted by the community that we have to achieve text modelling, given that changing the ontology at runtime
a better synergy between context-aware systems and their introduces potential issues with conflicts and ontology in-
users by providing control and intelligibility [8]. There is no tegrity. The Java Context Aware Framework (JCAF) [1]
general rule about the necessary level of control since in- takes an object-oriented approach and forwards the rea-
dividual users have varying preferences. As pointed out by soning of context rules to the application layer. This has the
Hardian [6], there needs to be a balance between the level effect that context rules are spread over various applica-
of control and automation. In order to provide end users tions which can lead to issues concerning conflict manage-
some control, visual programming tools such as the Jigsaw ment and inconsistent application behaviour across client
Editor [7] and iCAP [4] as well as commercial applications applications. In order to overcome this issue, JCOOLS [9]
such as IFTTT1 have been developed for defining simple “if integrates JCAF with the Drools inference engine. Although
situation then action” context rules. Note that in this case a JCOOLS is very promising, it lacks support for different ex-
situation consists of one or more events such as the current pertise levels and reusable user-defined situations. Finally,
temperature. Nevertheless, the complexity of context-aware the most well-known and used context-aware framework in
systems is significantly increased in new forms of smart the Human-Computer Interaction (HCI) community is the
homes. In such a complex environment, advanced mod- Context Toolkit by Dey et al. [2]. The framework applies
elling support for end users is required since simple context a component-based software design where components
rules cannot cover all desired smart home behaviour [10]. are represented by so-called widgets. Each widget pro-
cesses information, from low-level data input to higher-level
In this paper we explore the opportunities for advanced end abstract context information (i.e. detected situations). In
user modelling of rule-based context-aware applications. order to foster intelligibility and offer control over context
We propose a multi-layered context modelling approach rules, Dey and Newberger extended the Context Toolkit with
which supports the seamless transition between different a middleware that includes the concept of situations [3].
levels of expertise. We further enable the reuse of situa- Whenever all the widgets of a given situation are active,
tions defined by end users. Finally, we discuss an imple- the situation will invoke an action of the context services
mentation of the Context Modelling Toolkit (CMT) enabling listening for it. While some user control is provided via the
the practical use of our multi-layered context modelling ap- customisation of parameters, the reuse of situations in the
proach. definition of other situations is not possible.
Background We can conclude that there is a broad spectrum of context
For decades context-aware frameworks have been devel- modelling tools ranging from context-aware development
oped to facilitate the development of context-aware applica- frameworks to end-user tools. Nevertheless, existing con-
tions. They often include support for context modelling and text modelling solutions do not foresee a seamless transi-
reasoning. The SOCAM [5] framework is an example of an tion between the different levels of expertise of end users,
ontology-based context-aware framework. Unfortunately, expert users and programmers and they do not support the
1
reuse of user-defined situations.
https://ifttt.com
38
End User Multi-layered Context Modelling Approach fact or situation is only done when the template is used to
(3) In order to facilitate the end-user context modelling process, create a concrete situation or context rule. In order to know
Tool Rules
we foresee the possibility to let end users create new situa- which object types are needed by the template, it contains
tions such as ‘cooking’ by creating so-called situation rules a To Fill In part listing the required object types which
which take the form of “if situations then new situation”. In are connected to the corresponding input variables of sit-
(1)
its simplest form, the IF side of a situation rule can just be uations or functions on the IF side. When a user fills in a
a combination of context data as seen in existing visual template, they only have to provide the concrete instances
Programmer
programming tools, such as iCAP [4], in order to construct of the listed object types such as Person:Bob. Finally, the
context rules. Note that this context data includes facts (real THEN side of a template defines the new situation. Since sit-
world objects such as Person is Bob), events or situations. uations can have parameters, template designers can add
Situations Rules (2) Besides simple situation rules, we also support the reuse these to the new situation by defining which properties of a
of newly defined situations on the IF side. For example, template’s required object types have to be forwarded to the
Facts Actions the situation rule “IF Person is Bob and Location is kitchen new situation. When a template is created, it can be reused
THEN Bob is in the kitchen” takes the facts Person:Bob to define custom situations or context rules. For instance,
and Location:kitchen as context data on the IF side the situation rule “IF Person is Bob and Location is kitchen
Figure 1: Existing context
to define a new situation. This new situation can then, for THEN Bob is in the kitchen” could be created by using the
modelling approaches
example, be reused in the situation rule “IF Bob is in the PersonIsAtLocation template by simply filling in the vari-
kitchen AND stove is on THEN Bob is cooking” or in a con- ables Person:Bob and Room:kitchen and by naming the
End User
text rule such as “IF Bob is in the kitchen THEN turn radio new situation as “Bob is in the kitchen”. The new situation is
Situation
Filled in on”. In contrast to existing work, with our approach users do identified by its name. Similarly, the same template could be
(7)
Templates not have to re-define a situation each time they want to use used to define that Alice is in the living room.
Rule
(4) it in a different situation or context rule.
The situation rules and templates were introduced to sup-
Expert User
We further make use of templates to simplify the definition port users with different levels of expertise as proposed by
of situation and context rules. A template serves as a skele- Ur [10]. As illustrated in Figure 1, in existing systems facts
Templates (8)
ton for rules. The IF side can consist of multiple events, (e.g. Person is Bob), situations (e.g. cooking) and actions
(6) situations or logical functions. Similar to programming con- (e.g. turning on the lights) (1) are usually predefined by a
(5)
Programmer
cepts, a function evaluates some logical statements and programmer and can be applied by an end user to construct
returns the result. In our multi-layered context modelling so- simple “IF situation THEN action” context rules (3). Fur-
Situations Actions
lution, functions only return boolean values. Furthermore, thermore, in context modelling solutions without end-user
situations and functions can have a number of parameters. control, programmers also implement the desired context
Facts Functions Rules
For example, a function IsPersonInLocation evaluates rules (2). We extended the previous model with an inter-
whether the given Person is in the specified Room and re- mediate layer for expert users and integrated the notions of
Figure 2: Multi-layered context turns true or false. In contrast to the definition of a context custom and reusable situations as shown in Figure 2. Sim-
modelling approach rule, a template does not specify a specific fact or situation ilar to existing systems, in our multi-layered context mod-
for a parameter of a given type. The binding to a specific elling approach, programmers are responsible to provide
39
context data in the form of facts, situations and actions to Application & CMT Server
Rule the end user (4). Besides the previously mentioned context Sensors
Compiler
Database
(1,1) CMT-REST Interface
hasOutput (1,*)
data, they also have to provide functions (5) in order that
compiled Observer
to rule Template THEN-side
expert users can create templates (6). End users might fill CMT Client Library
in templates by using the provided context data. In addi- Translator
Delegator db4o
(1,1) (1,1) (1,*)
X tion to the templates defined by expert users, end users can CMT-RMI
Drools
Com Interface
also use some default templates such as an AND template Interface
Event
has Observer
hasInput Situation hasAction to construct simple situation and context rules. The com- Listener
Manager
pleted templates define new situation or context rules (7). Drools
(1,*) (0,*) (0,*) New situations are made available at the programmer layer
IF-side Situation Action in order that they can be reused at the expert and end-
user layer (8). Similarly, custom context rules flow from the Figure 4: CMT architecture
(0,*) (0,*)
end-user layer back to the programmer layer and can be
hasInput
used by programmers or other applications. This provides a currently support the Java Remote-Method Invocation (RMI)
Event
seamless transition between the three layers. and REST communication protocols. Since the two pro-
has
Function tocols require a different data exchange format, the CMT
Context Modelling Toolkit server contains the CMT-RMI and CMT-REST components.
(0,*) The proposed multi-layered context modelling approach has These components translate their input to CMT data model
(0,*) Context (1,1) been implemented in the Context Modelling Toolkit (CMT). entities and generate the required output format when data
Function Input
CMT has been designed as a client-server architecture and is sent to the clients. Finally, the Java-based CMT Client
(0,*) is implemented in Java. While the server side takes care of Library consists of a Translator component for trans-
has (0,*) the context reasoning, clients such as sensors, third-party lating Java objects to CMT data model entities and to the
Parameter Item
applications and applications which allow end users to de- corresponding JSON formats, the Com Interface which
fine new situations, provide the necessary context data to abstracts REST calls and the Listener component which
the server. As shown in Figure 4, CMT consists of a db4o2 provides Java listeners for CMT.
database backend and uses the non-persistent version of
Property Fact
the Drools 63 rule reasoning engine. In order to have a uni- Clients first have to register the type of situations or facts
(0,*) (1,1) fied representation of the context data delivered by clients, they will send to CMT. Since Drools requires compiled Java
has we have designed the conceptual CMT data model shown classes of the registered types, the Compiler component
Property has
Type FactType in Figure 3. The data model highlights the elements of the will compile these classes and add them to the classpath
(0,*) previously described multi-layered modelling approach in at runtime. After registration, clients can add instances of
(0,*)
the form of an Entity-Relationship (ER) diagram. The dif- the registered types to the server. In case the instance is a
(0,*) has
Type Context
Type
ferent entities of the CMT data model serve as a common fact, the Delegator component sends it to the Database
vocabulary used in any client-server communication. We Interface which inserts it into db4o. The Delegator
2
component further sends both, facts and situations, to the
Figure 3: CMT data model http://www.mono-project.com/archived/db4o
3
http://docs.jboss.org/drools/release/6.1.0.Final
Drools Interface where they are inserted in Drools and
40
Figure 5: Design of templates in the expert graphical user interface
rules are re-evaluated. Furthermore, new templates are which again has been defined by a programmer and returns
stored in the database. When a filled in template is re- true if there is no movement in a given room. Further,
ceived, it is compiled to the Drools DRL rule format. The she adds a time constraint which indicates that it must be
new rule is then inserted into the Drools knowledge base later than a specified time. When the logical statements are
and all rules are re-evaluated. Note that our implementation defined, Alice has to add the object types to be provided
of the rule compilation is a complex mechanism which takes when using the template. In our example, a Location and
into account the full first order logic and provides in depth ItIsAfter type are added to the To Fill In part. Next, she
error handling for client applications. When Drools detects connects the Location type to the inBed and noMovement
some conflicting rules, the Event Manager forwards them parameters to indicate that the entered location has to be
to the clients. In the future, we plan to further develop this passed to the two functions. Finally, she allocates the room
component to provide some intelligibility. to the new situation on the THEN side by clicking the ar-
row of the room field in the Location type. After saving the
Graphical User Interfaces template, Alice can use the template to define “Sleeping”
End users as well as expert users can interact with CMT situations such as that she or her son Bob sleep.
via a graphical user interface (GUI). The expert GUI en-
ables the design of templates as shown in Figure 5. In our The end user GUI shown in Figure 6 displays the created
proof of concept scenario Alice designs a template to cre- template with a placeholder for the required Location in-
ate “Sleeping” situations. First, she adds to the IF side the stance where Alice can enter Bob’s room. In addition, there
inBed situation which has been defined by a programmer is a ItIsAfter placeholder. By clicking the ‘Enter Values’
and triggers when a pressure sensor in the bed detects button, Alice can enter the desired time (e.g. 20:00). She
some pressure. Alice also adds the function noMovement then enters “Bob Sleeps” as the name for the new situation
41
on the Situation side. Similarly, she can enter her room and References
specify 22:00 to define that she sleeps. Finally, the sleep- [1] Jakob Eyvind Bardram. 2005. The Java Context
ing situations can be used to define context rules. Alice can Awareness Framework (JCAF) - A Service Infrastruc-
add “Bob Sleeps” and “I Sleep” to the IF side of the AND ture and Programming Framework for Context-Aware
template (i.e. default context rule template) and add the Applications. In Proc. Pervasive.
action that the night light has to be turned on to the THEN [2] Anind Dey, Gregory Abowd, and Daniel Salber. 2001.
side as shown in Figure 7. Note that similar to the creation A Conceptual Framework and a Toolkit for Supporting
of templates for new situations, expert users can design the Rapid Prototyping of Context-Aware Applications.
templates for advanced context rules. Human-Computer Interaction 16, 2 (2001).
[3] Anind Dey and Alan Newberger. 2009. Support for
Conclusion and Future Work Context-Aware Intelligibility and Control. In Proc. CHI.
We are currently investigating the further design of the ex- [4] Anind Dey, Timothy Sohn, Sara Streng, and Justin Ko-
Figure 6: End-user GUI to define new pert user and end user graphical user interfaces. Besides dama. 2006. iCAP: Interactive Prototyping of Context-
situations usability and interaction enhancements, we are exploring Aware Applications. In Proc. PerCom.
the integration of feedforward and intelligibility features. Fur- [5] Tao Gu, Xiao Hang Wang, Hung Keng Pung, and Da
thermore, we plan to deploy CMT under a research license Qing Zhang. 2004. An Ontology-based Context Model
in order that the HCI community can explore new opportuni- in Intelligent Environments. In Proc. CNDS.
ties in designing user-centric context-aware applications. [6] Bob Hardian, Jadwiga Indulska, and Karen Henrick-
sen. 2006. Balancing Autonomy and User Control in
We have presented a multi-layered context modelling solu- Context-Aware Systems - a Survey. In Proc. PerCom.
tion and the corresponding CMT data model as an innova- [7] Jan Humble, Andy Crabtree, Terry Hemmings, Karl-
tive approach for end users to control their context-aware Petter Åkesson, Boriana Koleva, Tom Rodden, and
applications. The presented solution contributes to the ex- Pär Hansson. 2003. "Playing with the Bits" User-
isting body of work by enabling end users to define new configuration of Ubiquitous Domestic Environments.
custom situations in the form of “if situations then new sit- In Proc. UbiComp.
uation” at runtime and to reuse these situations in other [8] Brian Lim and Anind Dey. 2009. Assessing Demand
situation and context rules. Ultimately, the proposed multi- for Intelligibility in Context-Aware Applications. In Proc.
layered context modelling approach enables end users, UbiComp.
Figure 7: End-user GUI to define new
expert users as well as programmers to collaboratively de- [9] Jongmoon Park, Hong-Chang Lee, and Myung-Joon
context rules by using the default AND
template velop their context models. Lee. 2013. JCOOLS: A Toolkit for Generating Context-
aware Applications with JCAF and DROOLS. Journal
Acknowledgements of Systems Architecture 59, 9 (2013).
The research of Sandra Trullemans is funded by the Agency
[10] Blase Ur, Elyse McManus, Melwyn Pak Yong Ho, and
for Innovation by Science and Technology in Flanders (IWT).
Michael Littman. 2014. Practical Trigger-Action Pro-
We would further like to thank Wouter Mensels and Brecht
gramming in the Smart Home. In Proc. CHI.
De Rooms for their work on an earlier version of the toolkit.