Workshop "From Objects to Agents" (WOA 2019) Towards an Ontology-Based Framework for a Behavior-Oriented Integration of the IoT Domenico Cantone, Carmelo Fabio Longo, Marianna Nicolosi-Asmundo, Daniele Francesco Santamaria, Corrado Santoro University of Catania Department of Mathematics and Computer Science Viale Andrea Doria, 6 - 95125 - Catania, ITALY {domenico.cantone, fabio.longo}@unict.it, {nicolosi, santamaria, santoro}@dmi.unict.it Abstract—We present a prototype version of an ontology-based Semantic web is a vision of the web in which machine- framework, called P ROF -O NTO, that integrates IoT devices and readable data allows software agents to query and manipulate users with domotic environments. P ROF -O NTO is based on a information on behalf of human agents. In such a vision, web novel OWL 2 ontology, called OASIS (Ontology for Agents, Systems, and Integration of Services), modelling behaviors of information carries explicit meaning, so it can be automatically agents such as IoT devices and users, and other informa- processed and integrated by agents, and data can be accessed tion concerning user requests, their executions, restrictions and and modified at a global level, thus resulting in increased authorizations. User requests are performed by automatically coherence and dissemination of information. Moreover, with selecting compatible devices: agents expose their behaviors and the aid of reasoners, it is possible to infer and process also are invoked accordingly to what they are able to do on specific categories of components. OASIS is also used to build semantic implicit information present in the data, thus gaining a deeper knowledge bases that operate as transparent communication and knowledge of the domain. Automated reasoning systems allow information exchange systems among agents. one to also verify the consistency of the model and query the data-set. The definition of a specific domain is widely called I NTRODUCTION ontology [1], [2]. Automation refers to the capability of devices to act on be- In this paper, we present a prototype version of P ROF - half of users in specific environments, with a little effort from O NTO,1 an ontological framework for the integration of users their part. Automation is not confined only to factories, farms, and IoT devices with domotic environments, which acts as a and cities, but also small environments such as homes can take home assistant. P ROF -O NTO exploits a novel ontology, called advantage of it. An ecosystem comprising interrelated devices OASIS (Ontology for Agents, Systems, and Integration of Ser- capable to transfer data over a network and cooperate without vices),2 which models user requests together with restrictions human control is widely called Internet of Things (IoT). IoT and scheduling, device behaviors, device authorizations imple- is massively used in smart environments, especially in home mented by smart contracts [3], and information concerning the automation systems. Domotic systems typically connect web- execution of user requests by devices. With respect to other enabled devices through internet to a central hub or assistant, Agent System paradigms such as Artifacts&Agents [4], OASIS which is responsible of managing them. adopts a general approach where agents are entities able to Currently, many domotic systems are available on the mar- perform actions, whereas components are subjected to actions ket. Often assistants and devices are strictly bound to their carried out by agents. providers and thus users are tied to such providers as well. It Transcriptions of user requests are mapped by a BDI rule- turns out that interchangeability of devices is hardly reachable based system, called PROFETA, in OASIS knowledge bases. without the intervention of third-part applications and, as a In the current version of the assistant, user requests are consequence, users may not install devices or assistants de- satisfied by automatically selecting devices whose behaviors ployed by other sellers. Besides specific marketing strategies, fulfill such requests. Communication between the assistant and several obstacles prevent the interchangeability of devices. the devices depends on a specific knowledge base of OASIS, Among them, the most relevant one concerns connectivity, used as a communication protocol, that specifies the device to networking, and communication protocols whose usage largely be activated, the action to be performed, and the recipient of depends on the specific IoT applications deployed, which the action. are to be regarded as black boxes. Moreover, it is almost The paper is structured as follows. Section I provides impossible to determine a priori what a device is capable to an overview of the semantic web, introducing ontologies do within the environment and how its functionality can be and their related modelling languages. Section II deals with controlled by users. Such problems could be solved if devices related works. Section III presents our proposed ontological were selected on the basis of what they are able to do through open and shared knowledge bases and if they communicated 1 https://github.com/dfsantamaria/ProfOnto.git via a common, transparent protocol. 2 http://tiny.cc/OASIS-Ontology 119 Workshop "From Objects to Agents" (WOA 2019) framework and the way in which it is exploited to extract In [12], a comprehensive ontology for representing IoT meaningful data. Section IV describes a case-study that shows services is presented, together with a discussion on how it how our proposed framework can be used in a real application. can be used to support tasks such as service discovery, testing, Finally, Section V concludes the paper with some hints for and dynamic composition, taking into account also parameters future work. such as Quality of Services (QoS), Quality of Information (QoI), and IoT service tests. I. P RELIMINARIES A unified semantic knowledge base for IoT, capturing the Applications that automatically process information, instead complete dynamics of IoT entities and where their heterogene- of just presenting it, and exchange information with other ity is hidden and semantic searching and querying capabilities applications need appropriate languages, with formally defined are enabled, is proposed in [13]. syntax and semantics. This issue turns out to be particularly Unification of the state-of-the-art architectures, as put for- relevant for the web and, in general, for any distributed envi- ward by the scientific community of the Semantic Web of ronment. The Word Wide Web Consortium (W3C) recommends Things (SWoT), by means of an architecture based on different the Web Ontology Language (OWL), a family of knowledge abstraction levels, namely Lower, Middle and Upper Node representation languages relying on Description Logics (DLs) (LMU-N), is described in [14]. The LMU-N architecture [5], as a solution to this problem and indicates it as a standard provides a reading grid used to classify processes, to which for representing ontologies. Ontologies are formal descriptions the SWoT community contributes, and to describe how the of the domain of interest, defined by combining three basic semantic web impacts the IoT. syntactic categories: entities, expressions, and axioms. In [15], an ontology providing an elementary approach to OWL, currently in version 2.1, provides users with con- modelling agents’ behaviors and artifacts is proposed together structs useful for the design of ontologies in real-world with a tool that uses the ontology to generate programming domains that are not available in the basic semantic web code for agent-oriented software engineering. model Resource Description Framework (RDF) and in the None of the aforementioned work deals in depth with agent basic semantic web language RDF Schema (RDFS). As RDF, behaviors or tries to formalize the interaction mechanism OWL 2 is grounded on the idea of triples or statements, among them. As far as we know, this paper represents the first each one representing an atomic unit. Triples are ways to attempt of applying semantic web technologies as a commu- connect two entities or an entity and a data-value, each one nication protocol among IoT devices and as a representation represented by an Internationalized Resource Identifier (IRI), system for their behaviors and interactions. i.e., a sequence of characters that unambiguously identifies a resource within a specific context. Entities represent the III. T HE F RAMEWORK P ROF -O NTO primitive terms of an ontology and are identified in a unique way. They are individuals (actors), object- and data-properties In this section, we describe the most important features (actions), and classes (sets of actors with common features). In of the ontology OASIS, and the software architecture imple- order to provide a formal description of the domain, OWL 2 menting the prototype version of the ontology-based domotic triples can be organized into two main categories: expressions assistant called P ROF -O NTO. and axioms. Expressions are obtained by applying OWL 2 constructs to entities to form complex descriptions, whereas A. The ontology OASIS axioms describe what is true in the domain.3 OASIS is an OWL 2 ontology consisting of about 120 To retrieve and manipulate semantic knowledge, the W3C classes, more than 100 object-properties, 2 data-properties, and recommends the SPARQL query language as the standard more than 900 axioms. Among other information, it currently query protocol for RDF. Like SQL, SPARQL is a declarative models: a) behaviors of agents (i.e., users and devices) in query language to perform operations on data represented as terms of the operations that they are able to perform; b) agent a collection of RDF triples. A SPARQL query has a head configurations; c) agent requests; d) executions of operations and a body: the head comprises a modifier identifying the and their related status. corresponding type of query, whereas the body consists of an RDF triple pattern. The reader is referred to [7] for a detailed overview of SPARQL. Agent GoalPart TaskPart II. R ELATED WORK subclass of subclass of subclass of dependsOn dependsOn In the last decade, integration of agent systems and ontolo- hasBehavior Goal Task Device Behavior Description Description gies has been deeply studied in several contexts [8], [9], [10]. consistsOfGoalDescription consistsOfTaskDescription Concerning IoT, ontological approaches have been focused hasTaskObject hasTaskOperator hasTaskParameter mainly on sensors, with the purpose of collecting data for TaskObject Task TaskOperator generating perceptions and abstractions of the world [11]. Parameter 3 For a detailed explanation of axioms and expressions introduced in OWL 2, the reader is referred to [6]. Figure 1. Ontology schema of device behaviors 120 Workshop "From Objects to Agents" (WOA 2019) Goal Task Enviromental Agent Behavior TaskObject Type Description Description Component hasTaskOperator hasBehavior LightAgent_ hasTaskObject hasType SwitchOn LightAgent SwitchOn SwitchOnTask Bulb Description turn_on Behavior consistsOfTaskDescription consistsOfGoalDescription Y Y Y TaskOperator hasBehavior light_object turn_off LightAgent_ hasTaskObject _type SwitchOff SwitchOff SwitchOffTask Description Behavior consistsOfGoalDescription consistsOfTaskDescription Class hasTaskOperator Instance Legenda Figure 2. A representation of a smart bulb device in OASIS Despite the fact that OASIS is able to represent behaviors consistsOfGoalDescription. of generic agents, the ontology is mainly focused on devices The core of OASIS revolves around the description of entrusted by users to perform certain operations within a atomic operations introduced by the instances of the class domotic context. Behaviors of such agents are described by TaskDescription. Atomic operations are the most simple ac- the schema in Figure 1. tions that agents are able to perform and that they expose to The main classes of OASIS and their characteristics for other agents. Hence, atomic operations represent what agents describing behaviors of agents are summarized next. can do and what they can ask other agents to do by means - Agent: comprises all the individuals capable of executing of request submissions. Instances of the class TaskDescrip- actions on the associated components. This class includes, tion are related with three elements that uniquely identify among others, the classes HumanAgent, SoftwareAgent, the operation. The first element is an instance of the class and Device, mapping physical people, software or pro- TaskOperator, characterizing the action to perform. Instances grams, and physical devices, respectively. The class Hu- of TaskDescription are related with instances of TaskOperator manAgent contains, in its turn, the class User, represent- by means of the object-property hasTaskOperator. The second ing users that usually access the system. element is an instance of the class TaskObject, representing - TaskDescription: describes atomic operations (e.g., turn the object recipient of the actions (described by the instances on, turn off, wipe, and so on) that an agent performs the class TaskOperator) performed by devices. Instances of as a result of some requests made by other agents. An TaskDescription are related with instances of TaskObject by atomic operation O may depend on other atomic opera- means of the object-property hasTaskObject. The third element tions, whose execution is mandatory in order to perform is an instance of the class TaskParameter. It is conceived for the operation O. Dependencies of atomic operations are complex actions requiring a specific input parameter in order modelled through the object-property dependsOn, which to accomplish the requested operation, such as the temperature relates instances of the class TaskDescription. of air conditioners or the light intensity of bulbs. Instances of - GoalDescription: models a set of atomic operations (rep- TaskDescription are related with instances of TaskParameter resented by the class TaskDescription), whose execu- by means of the object-property hasTaskParameter. tions are subject to no order. Execution dependencies of Objects and parameters are related by means of the object- goals (non-atomic operations) are modelled through the property hasType with instances of the class Type. The latter object-property dependsOn, relating instances of the class class comprises a set of categories that specialize the type GoalDescription. Instances of the class GoalDescription of elements introduced in OASIS. For example, floors of are linked to the related task descriptions by means of buildings are of type “floor” while bulbs are of type “light the object-property consistsOfTaskDescription. object”. - Behavior: represents the behavior of a single agent in Figure 2 describes a smart bulb device through its behavior. terms of what the agent is able to do. It comprises a In particular, the device shows two behaviors, each one con- set of non-atomic operations (described by instances of sisting of a single goal that, in its turn, consists of a single task. the class GoalDescription) whose execution is subject to The device is able to perform the actions of turning on and no order. Instances of the class Behavior are linked to off (i.e., the task operators) the connected bulb (i.e., the task the related goal descriptions through the object-property object). The latter component is represented by the individual 121 Workshop "From Objects to Agents" (WOA 2019) bulb. defined by the device in its behavior description by means Devices are configured in order to fit the needs of some of the object-property setUpFrom. agents such as users. A device configuration comprises a An example of a component configuration is shown in set of optional features that users associate with a single Figure 4. The example extends the smart bulb device illustrated component (recipient of some device operations) or with a in Figure 2 by providing a configuration, introduced by the single device. There are many features that can be associated user Alan, of the single bulb controlled by a device. Alan with objects or devices such as virtual collocation, physical specifies the kitchen as physical position of the bulb; in this position, nickname, and so on. A specific configuration pro- case, the object-property hasSpaceSpan is used to link the viding information about how devices communicate can be configured object to an instance of the class Space representing assigned to devices when they are automatically detected by Alan’s kitchen. In this specific smart bulb example, the device the system. is physically indistinguishable from the controlled bulb. In Configurations are represented in OASIS according to the such a case, the user configuration should be provided for the schema in Figure 3, and make use of the following classes: bulb as shown in Figure 4, leaving to the device configuration - Configuration: models configurations admitted in OA- only the management of the connection information. SIS. Agents that create the configuration of a device or of a - ComponentConfiguration: is a subclass of the class Con- component are specified by the object-property configura- figuration and models configurations associated with the tionProvidedBy, linking an instance of the class Configuration components of the environment involved as objects in to an instance of the class Agent. some operations (i.e., instances of the class TaskObject). As stated above, OASIS models user requests consisting in - DeviceConfiguration: models configurations associated entrusting some devices to do something within the domotic with devices. environment, in accordance with the restrictions imposed by - Connection: specifies how to physically communicate their configurations. User requests are introduced by exploiting with a device, e.g., the protocol used and the commu- classes and properties used to model device behaviors, except nication address associated with the device. that the desired sets of user goals to be accomplished are related with a user plan. The ontology schema for user requests is depicted in Figure 5. Component setUpFrom Connection connectsTo User GoalPart TaskPart hasConfiguration hasConnection requests subclass of subclass of setsUp Component Device Configuration dependsOn dependsOn Configuration subclass of subclass of Configuration Plan Goal Task Description consistsOfGoalDescription Description consistsOfTaskDescription Description configurationProvidedBy hasConfiguration hasTaskObject hasTaskOperator hasTaskParameter subclass of Agent Device TaskObject Task TaskOperator Parameter Figure 3. Ontology schema of device configurations Figure 5. Ontology schema of user requests Instances of the class Device are associated with instances User requests are introduced by instances of the class Plan- of the class DeviceConfiguration by means of the object- Description related to instances of the class GoalDescription. property hasConfiguration. Instances of DeviceConfiguration The user requesting the action is modelled by instances of are related with instances of the class Connection by means of the class User that are linked to the plan through the object- the object-property hasConnection. As stated above, instances property requests. Descriptions of goals from user requests are of the latter class are exploited for creatiing an entry-point modelled analogously to the descriptions of goals from device that puts devices in communication. In such a case, the object- behaviors. property connectsTo is used to link instances of DeviceCon- In Figure 6, we show an example of a user request consisting figuration with instances of Connection. of turning off a light, performed by the user Alan. The object-property hasConfiguration is also used to link Finally, a user request is associated with an attempt of instances of the class Component with instances of the class finding a device that is able to execute it. OASIS uses the class ComponentConfiguration. Once a component has been con- TaskExecution for representing the attempt of executing an figured, the instance of ComponentConfiguration is related by agent request. Instances of the class TaskDescription defined means of the object-property setsUp with a fresh individual by the user requests are related with instances of the class representing the configured component. The latter individual, TaskExecution through the object-property hasTaskExecution. in its turn, is related with the non-configured component An instance of Device, representing the device responsible 122 Workshop "From Objects to Agents" (WOA 2019) Enviromental Component Task TaskObject Type Space User Type Component Component Configuration Description hasTaskObject hasType hasConfiguration light_object_ Alan-Light- SwitchOnTask Bulb User-Alan type Configuration configurationProvidedBy Y setsUp setUpFrom hasType Class Alan-Light Main_Kitchen kitchen hasTaskObject hasSpaceSpan SwitchOffTask Instance Legenda Figure 4. Example of configuration in OASIS of satisfying the request, is related with the instance of In Figure 8, we show an example concerning the execution TaskExecution by means of the object-property performs. Once of Alan’s request depicted in Figure 6. In the example, Alan’s the selected device attempts to fulfill a user request, the status request of turning off the light, described in Figure 6, is of the execution task is updated by linking the instance of accomplished by the smart bulb (represented in Figure 2) that TaskExecution to an instance of the class ActionStatus. The sets to succeeded the execution status of the action. most important instances of ActionStatus are the individuals succeeded status, representing the correct execution of the Task Task Device Class Description Execution action, and the individual failed status, representing the status of those actions that have not been accomplished. The mod- Instance elling of the attempt of fulfilling agent requests in OASIS is Legenda alan-task- hasTaskExecution 1-1-1 summarized in Figure 7. alan-task- performs 1-1-1-exec Light-Agent hasTaskObject hasTaskOperator Plan Goal Task hasStatus User Description Description Description Alan-Light turn_off succeeded_ status ActionStatus performs consistsOfTaskDescription User-Alan alan-plan1 alan-goal-1-1 Task Task consistsOfGoalDescription Object Operator alan-task-1-1-1 hasTaskObject hasTaskOperator Figure 8. Ontology schema of a task execution hasType light-object-type alan-task-object turn_off Class B. Software Architecture Instance In this section, we briefly summarize the main features of Type Task Object Task Legenda the architecture of P ROF -O NTO, illustrated in Figure 9. Operator Figure 6. Example of modelling of user requests OASIS KBBehavior ONTOLOGY PROFETA hasStatus Task Task ActionStatus KBBelief Device Execution Description performs hasTaskExecution HERMIT DL REASONER hasTaskObject hasTaskOperator hasTaskParameter TaskObject Task TaskOperator Parameter Figure 9. Architecture of P ROF -O NTO The core of P ROF -O NTO comprises by OASIS (the ontolo- Figure 7. Ontology schema of execution of user requests gies illustrated in Figures 1, 3, 5, and 7), the dataset KBBe- 123 Workshop "From Objects to Agents" (WOA 2019) havior collecting behaviors and user defined configurations of - Device connection and configuration. When a device devices (RDF graphs of the types depicted in Figures 2 and tries to connect to P ROF -O NTO, PROFETA receives 4), and the data-set KBBelief containing user requests together the request and generates the corresponding knowledge with their execution information (RDF graphs of the types base of OASIS. This is passed to the ontological core reported in Figures 6 and 8). of P ROF -O NTO, which in turn updates KBBehavior ac- P ROF -O NTO knowledge bases are implemented in JAVA by cordingly and executes the HermiT reasoner to check its exploiting the OWL API [16] and Apache Jena [17], both consistency. used to manipulate the ontological information and to perform - User requests. When a user requests an action, PROFETA SPARQL queries. Consistency of P ROF -O NTO knowledge analyzes the command and produces the corresponding bases is checked by means of the HermiT DL reasoner [18]. RDF graph, which is submitted to the ontological core P ROF -O NTO also exploits PROFETA [19], a Belief-Desire- of P ROF -O NTO. P ROF -O NTO builds and performs the Intention (BDI) rule-based system able to trigger rules rep- related SPARQL queries. The result is then sent to resenting computations to be performed, in order to parse, PROFETA, which activates the selected device. If the interpret, and manage user requests and device connections. request can be accomplished, the data-set KBBelief is BDI rules are passed to the P ROF -O NTO core as OASIS updated accordingly. Once the device has performed the knowledge bases. action, it updates PROFETA by sending the execution Since most interaction among common users and IoT do- status. Then, PROFETA, in its turn, sends a request to mains occur through a natural spoken language, P ROF -O NTO the ontological core of P ROF -O NTO, which updates the manages user utterances via the PROFETA interface. Given data-set KBBelief. Finally, the HermiT reasoner is called a domotic command either by Speech-To-Text (STT) services by P ROF -O NTO to check the consistency of KBBelief. or Chatbots, PROFETA processes the string representing the IV. C ASE -S TUDY transcription of the user intention in natural language by implementing a robust dependency parser able to deal very We describe a simple case-study illustrating the working satisfactorily with imperative verbs (without raising the issues basics of the framework P ROF -O NTO. In our example, the treated in [20]), within a so-called Translation Service (TS). A environment consists of (a) a smart bulb, called light-device, domotic command usually has the form of an imperative verbal (b) a user, called Alan, and (c) a domotic assistant running phrase; consequently, the relations generated by the depen- P ROF -O NTO. dency parser and providing information about user intentions As a first step, light-device is connected to the assistant have the form dobj(arg1 , arg2 ), namely a direct object, where and suitably configured by the user. The device exposes its arg1 and arg2 are verb and object related to the intention, behavior by submitting the RDF graph illustrated in Figure respectively. For example, given the sentence wipe the floor, 10 to the assistant, which integrates it with the P ROF -O NTO PROFETA produces the relation dobj(wipe, floor). knowledge base as described in Section III. We denote with prof the prefix of OASIS and with dev the prefix of the smart Additionally, PROFETA supports other types of relation bulb ontology. such as pobj (preposition object), which provides additional Before fully connecting the smart bulb to the assistant information about the physical location of an object, and and integrating it with the domotic environment, the user is compound/prt (compound/particle), which provides the com- summoned to configure the device. In the particular case of positionality of phrasal verbs such as “turn off” or of nouns the light-device specification, the user may provide a friendly such as “living room”. The reader can refer to [21] for further name and indicate the physical collocation of the device. In details concerning the process of extraction of intentions from our example, the user Alan sets the name of the device to natural spoken language. main kitchen light and collocates it in the main kitchen. Once The result of the TS module consists of a set of beliefs such information is provided, the configuration manager maps representing user intentions together with related parameters. Alan’s request into an RDF graph (see Figure 11) and transmits Such beliefs are processed by the PROFETA engine through it to the assistant. the use of rules of the form: Subsequently, information provided by the smart bulb be- havior and the user configuration is merged with the KBBe- +Int(Verb,Obj,Loc) ≫ generate request(Verb,Obj,Loc), havior knowledge base. Such task is carried out by adding the triples to the KBBehavior knowledge base and by executing which produce the desired plan from the set of beliefs obtained the HermiT reasoner for consistency checking. from previous steps. The plan generate request consists in Once a device is connected to the system, namely once RDF sending user requests to the ontological core of P ROF -O NTO triples describing its behavior and configuration are integrated through an appropriate wrapper that maps plans in OASIS in the KBBehavior knowledge base, the system is ready to knowledge bases of the forms described in Section III-A. For accept user requests, which are formalized as RDF graphs as the sake of conciseness, details about the syntax of PROFETA well. Request RDF graphs are then merged with the KBBelief are omitted and can be found in [19]. knowledge base and the HermiT reasoner is executed. In the P ROF -O NTO takes care of the following two activities: smart bulb example, Alan sends the command “turn off the 124 Workshop "From Objects to Agents" (WOA 2019) light in the kitchen” to the assistant which generates the RDF The graph returned by the CONSTRUCT query in Figure graph illustrated in Figure 12. 13 is illustrated in Figure 14. CONSTRUCT { dev:LD-turnoff- prof:hasBehavior prof:hasBehavior dev:LD-turnon- ?selected agent prof:performs :alan-task1-1-exec . dev:light-device behavior behavior :alan-task1-1 prof:hasTaskExecution :alan-task1-1-exec . :alan-task1-1-1-exec prof:hasTaskOperator ?operation . prof:consistsOfGoalDescription prof:consistsOfGoalDescription :alan-task1-1-1-exec prof:hasTaskObject ?user object . ?user object prof:setUpFrom ?device object . } dev:LD-turnon- dev:LD-turnoff- goalDes goalDes WHERE { :alan-task1-1-1 prof:hasTaskOperator ?operation . prof:consistsOfTaskDescription prof:consistsOfTaskDescription :alan-task1-1-1 prof:hasTaskObject ?object . ?object prof:hasSpaceSpan ?space . dev:LD-turnoff- prof:hasTaskObject dev:LD-turnon- ?object prof:hasType ?obtype . taskDes taskDes ?space prof:hasType ?spacetype . prof:hasTaskOperator prof:hasTaskOperator ?config rdf:type prof:ComponentConfiguration . dev:ld-light1 ?config prof:configurationProvidedBy :user . ?config prof:setsUp ?user object . prof:turn_off prof:turn_on ?user object prof:hasType ?obtype . ?user object prof:hasSpaceSpan ?space user . ?space user prof:hasType ?spacetype . ?user object prof:setUpFrom ?device object . Figure 10. The RDF graph mapping the smart bulb behavior ?selected agent prof:hasBehavior ?behav . ?behav prof:consistsOfGoalDescription ?goal . ?goal prof:consistsOfTaskDescription ?task . prof:configurationProvidedBy ?task prof:hasTaskOperator ?operation . prof:hasConfiguration base:alan-ld1- ?task prof:hasTaskObject ?agent subject . base:alan config ?agent subject prof:hasType ?obtype . } base:ld1-light1 prof:setsUp Figure 13. The SPARQL CONSTRUCT query for Alan’s request base: base:alan- main_kitchen prof:kitchen prof:setUpFrom kitchen _light prof:hasSpaceSpan prof:hasType base: Figure 11. The RDF graph representing the configuration of the smart bulb base:alan-task1-1-1 dev:ld-light1 alan-task1-1-1-obj prof:setUpFrom prof:hasTaskExecution prof:hasTaskObject base:alan-task1- base:alan-plan1 base:alan-goal1-1 dev:light-device prof:turn_off 1-1-exec prof:hasTaskOperator prof:performs prof:consistsOfGoalDescription prof:consistsOfTaskDescription Figure 14. The RDF graph obtained by executing the query of Figure 13 prof:turn_off base:alan-task1-1-1 prof:kitchen hasTaskOperator prof:hasTaskObject prof:hasType By means of the graph in Figure 14, the smart bulb can execute the requested action, thus turning off the light. In fact, base: base: prof:light_object_type prof:hasType alan-task1-1-1-obj alan-task1-1-1- kitchen the agent committed to execute the action (dev:light-device), prof:hasSpaceSpan the operation (prof:turn off ), and the object of the action (dev:ld-light1) have been correctly spotted and the device can Figure 12. The RDF graph of the request of turning off the kitchen light be activated by sending to it the requested information. Subsequently, the device updates the status of the execution The RDF graph representing Alan’s request is analyzed by task by sending to the assistant the triple: P ROF -O NTO in order to produce a SPARQL CONSTRUCT :alan-task-1-1-1-exec prof:hasStatus prof:succeeded status . query to be executed over the knowledge base KBBehavior and over Alan’s request graph. The CONSTRUCT query produces if the task has been successfully accomplished, or by sending an RDF graph that associates Alan’s request with a specific the triple device able to fulfill his request. The SPARQL query obtained :alan-task-1-1-1-exec prof:hasStatus prof:failed status . from Alan’s command, consisting in turning off the kitchen if the task has not been performed for some reason. The light, is illustrated in Figure 13. The body of the query consists assistant updates its belief knowledge base by adding the of three parts. The first one is taken from the specifications of information relative to the execution status, as provided by Alan’s request. The second part, obtained from the ontology the device. Afterwards, the assistant is ready to accept new reported in Figure 3, explores all the user configurations in requests, configurations, or to connect new devices. order to discover the type of the device and the place the device has been installed in. The last part, constructed from V. C ONCLUSIONS AND F UTURE W ORK the ontology in Figure 1, selects an available device fulfilling In this paper we presented P ROF -O NTO, a prototype frame- the conditions specified in the first two parts. work that integrates users and IoT devices within domotic 125 Workshop "From Objects to Agents" (WOA 2019) environments. P ROF -O NTO is based on a novel ontology [7] B. DuCharme, Learning SPARQL. O’Reilly Media, Inc., 2011. called OASIS, modelling device behaviors, user requests, and [8] J. Hendler, “Agents and the semantic web,” IEEE Intelligent Systems, their executions. P ROF -O NTO acts in two phases. The first vol. 16, no. 2, pp. 30–37, 2001. phase consists in connecting devices to the domotic assistant. [9] M. Hadzic, E. Chang, and P. Wongthongtham, Ontology-Based Multi- Agent Systems. Springer Publishing Company, Incorporated, 2014. Devices share their behaviors by means of knowledge bases [10] D. Fritzsche, M. Grüninger, K. Baclawski, M. Bennett, G. Berg-Cross, of OASIS, which are automatically integrated with P ROF - T. Schneider, R. Sriram, M. Underwood, and A. Westerinen, “Ontology O NTO and whose consistency is checked by the HermiT DL Summit 2016 Communique: Ontologies within semantic interoperability ecosystems,” Applied Ontology, vol. 12, no. 2, pp. 91–111, 2017. reasoner. In the second phase, users send their requests to [11] G. Bajaj, R. Agarwal, P. Singh, N. Georgantas, and V. Issarny., “A study a BDI rule-based system, called PROFETA, that maps the of existing Ontologies in the IoT-domain,” hal-01556256, 2017. transcriptions of user requests in OASIS knowledge bases. [12] W. Wang, S. De, R. Toenjes, E. Reetz, and K. Moessner, “A Com- prehensive Ontology for Knowledge Representation in the Internet of In the last phase, P ROF -O NTO automatically selects devices Things,” in 11th International Conference on Trust, Security and Privacy compatible with user requests by means of SPARQL queries in Computing and Communications, IEEE, 2012. specifically constructed. Then, the resulting action is sent to [13] S. N. Akshay Uttama Nambi, C. Sarkar, R. V. Prasad, and A. Rahim, “A Unified Semantic Knowledge Base for IoT,” in World Forum on Internet the selected device in order to perform the required action. In of Things (WF-IoT), IEEE, 2014. this phase, a suitable knowledge base of OASIS is used as a [14] N. Seydoux, K. Drira, N. Hernandez, and T. Monteil, “Capturing the communication and information exchange system between the Contributions of the Semantic web to the IoT: a Unifying Vision,” in assistant and the selected device. Semantic Web technologies for the Internet of Things, ISWC, 2017. [15] A. Freitas, R. H. Bordini, and R. Vieira, “Model-driven engineering of We plan to integrate with P ROF -O NTO temporal modifiers, multi-agent systems based on ontologies,” Applied Ontology, vol. 12, action restrictions, conditionals already modelled by OASIS, pp. 157–188, 2017. and with user requests directly entrusting specific devices with [16] M. Horridge and S. Bechhofer, “The OWL API: A Java API for Working with OWL 2 Ontologies,” in Proceedings of the 6th International Con- the execution of actions. ference on OWL: Experiences and Directions - Volume 529, OWLED’09, We also intend to study how OASIS can be exploited by pp. 49–58, CEUR-WS.org, 2009. OntologyBeanGenerator 5.0 [22] inside the JADE framework [17] Apache Software Foundation, “Apache Jena, A free and open source Java framework for building Semantic Web and Linked Data applica- [23] to generate code for agents and artifacts and how it can tions,” 2011. https://jena.apache.org. exploited by CArtAgO [24], a framework for building shared [18] B. Glimm, I. Horrocks, B. Motik, G. Stoilos, and Z. Wang, “HermiT: computational worlds. An OWL 2 Reasoner,” Journal of Automated Reasoning, vol. 53, no. 3, pp. 245–269, 2014. We shall integrate OASIS with the ontology for IoT ser- [19] L. Fichera, F. Messina, G. Pappalardo, and C. Santoro, “A Python vices defined in [12] and the Sensor Ontology in [25]. In Framework for Programming Autonomous Robots Using a Declarative addition, we shall extend the set of actions and parameters Approach,” Sci. Comput. Program., vol. 139, pp. 36–55, 2017. [20] T. Hara, T. Matsuzaki, Y. Miyao, and J. Tsujii, “Exploring Difficulties provided by OASIS with the synset introduced by WordNet in Parsing Imperatives and Questions,” in Proc. of the 5th International [26], in order to make the whole infrastructure multi-language Joint Conference on Natural Language Processing, AFNLP, 2011. and meaning-oriented as in the case of [21]. [21] C. F. Longo, C. Santoro, and F. F. Santoro, “Meaning Extraction in a Domotic Assistant Agent Interacting by means of Natural Language,” In addition, we intend to define a set-theoretic representation in 28th IEEE International Conference on Enabling Technologies: of OASIS in the flavour of [27]. However, since OASIS Infrastructure for Collaborative Enterprises, IEEE, 2019. contains existential restrictions, we also need to modify the [22] D. Briola, V. Mascardi, and M. Gioseffi, “OntologyBeanGenerator 5.0: Extending Ontology Concepts with Methods and Exceptions,” in underlying set-theoretic fragment in such a way as to allow Proceedings of the 19th Workshop ”From Objects to Agents”, Palermo, a restricted form of the composition operator. The related Italy, June 28-29, 2018., pp. 116–123, 2018. reasoning procedure will then be adapted to the new set- [23] F. L. Bellifemine, G. Caire, and D. Greenwood, Developing Multi-Agent theoretic fragments exploiting the techniques introduced in Systems with JADE. Wiley, 2007. [24] A. Ricci, M. Piunti, M. Viroli, and A. Omicini, “Environment Program- [28], [29] in the area of relational dual tableaux. Finally, ming in CArtAgO,” in Multi-Agent Programming: Languages, Tools and we intend to replace the HermiT reasoner in P ROF -O NTO Applications, (Boston, MA), pp. 259–288, Springer US, 2009. with a suitably extended version of the set-theoretic reasoner [25] Word Wide Web Consortium, “Semantic Sensor Network Ontology.” https://www.w3.org/TR/vocab-ssn/. described in [30]. [26] Princeton University, “WordNet, A Lexical Database for English,” 2010. https://wordnet.princeton.edu. R EFERENCES [27] D. Cantone, C. Longo, M. Nicolosi-Asmundo, and D. F. Santamaria, [1] D. Oberle, N. Guarino, and S. Staab, What is an ontology? Handbook “Web Ontology Representation and Reasoning via Fragments of Set on Ontologies. Springer, 2009. Theory,” in Cate, B. and Mileo, A. (eds) Web Reasoning and Rule [2] T. Hofweber, Logic and Ontology. Edward N. Zalta (ed.), The Stanford Systems. Lecture Notes in Computer Science, vol. 9209. Springer, 2015. Encyclopaedia of Philosophy (Summer 2018 Edition), 2018. [28] D. Cantone, M. Nicolosi-Asmundo, and E. Orłowska, “Dual tableau- [3] N. Szabo, “Formalizing and securing relationships on public networks,” based decision procedures for some relational logics,” in Proceedings First Monday, vol. 2, no. 9, 1997. of the 25th Italian Conference on Computational Logic, CEUR-WS Vol. [4] A. Ricci, M. Viroli, and A. Omicini, “Give Agents Their Artifacts: The 598, Rende, Italy, July 7-9, 2010, 2010. A&A Approach for Engineering Working Environments in MAS,” in [29] D. Cantone, M. Nicolosi-Asmundo, and E. Orłowska, “Dual tableau- Proceedings of the 6th International Joint Conference on Autonomous based decision procedures for relational logics with restricted composi- Agents and Multiagent Systems, AAMAS ’07, (New York, NY, USA), tion operator,” Journal of Applied Non-Classical Logics, vol. 21, no. 2, pp. 150:1–150:3, ACM, 2007. pp. 177–200, 2011. [5] F. Baader, I. Horrocks, C. Lutz, and U. Sattler, An Introduction to [30] D. Cantone, M. Nicolosi-Asmundo, and D. F. Santamaria, “A set- Description Logic. Cambridge University Press, 2017. based reasoner for the description logic DL4, × D ,” Proceedings of SETS [6] D. Allemang and H. J., “Semantic Web for the Working Ontologist: 2018, Southampton, United Kingdom, 5 June 2018. CEUR Workshop Effective Modeling in RDFS and OWL,” Elsevier, 2011. Proceedings, ISSN 1613-0073, vol. 2199, 2018. 126