We present the Simple Planning Annotation (spa) ontology for modeling preconditions and efects of interaction afordances within Web of Things Thing Descriptions and propose a mapping to the Planning Domain Definition Language to enable robot-device interaction. We use encoded semantic knowledge to generate a planning problem that can be used within existing AI planning algorithms to dynamically plan interaction sequences to achieve specified goals without the extensive pre-programming traditionally required, as demonstrated by the validation of our approach through a prototypical implementation. The scalability evaluation shows that when the number of input Web of Things actions increases by a factor of 1, 000, the runtime of the implemented prototype increases by about 14.3% and the memory consumption by about 9.4%, indicating vertical scalability.
CEUR ceur-ws.org
Autonomous robotic agents can perform tasks by interacting and manipulating their
environment either through direct physical interaction, such as fetch-and-place and wiping tasks [
Robotic agents can parse TDs, but TDs lack information about preconditions and efects of
interactions. This limitation prevents robotic agents from reasoning out the correct order of
command execution using AI planning algorithms to achieve predefined goals, especially in
unfamiliar environments. As a result, robotic agents rely on control programs that specify their
interactions with other devices and outline step-by-step procedures for each task [
By incorporating a semantic description of the conditions required to invoke interactions
and the efects of those interactions on devices into TDs, and by providing a mapping algorithm
between the semantic information contained in TDs and the standardized Planning Domain
Definition Language (PDDL) [
Previous approaches attempting to integrate information about sequential behavior, preconditions, and efects into TDs have primarily focused on interactions between resource-constrained devices that are unable to use planning algorithms. Therefore a pre-computed interaction plan has been annotated in TDs, which hinders use in dynamic scenarios. Other approaches have adopted centralized architectures where all environmental information is consolidated in a central knowledge repository. While this strategy is viable for mostly static smart home applications, it is not suitable for dynamic environments.
In this paper we present the spa ontology for modelling preconditions and efects of
interaction afordances in TDs. Together with the ontology, we introduce a mapping between
the semantic information contained in TDs into PDDL domain and problem definitions. The
mapping allows standard AI planning algorithms to plan the interactions necessary for an
autonomous robotic agent to achieve a given goal. We also provide a prototypical implementation
of the mapping algorithm with link following [
The contributions of our work are • an ontology describing preconditions and efects in TDs, • a mapping between semantic annotations in TDs and PDDL domain and problem statements, • and a validation via a prototypical implementation of a mapping algorithm with an evaluation of the scalability of the proposed mapping approach.
In the following sections, we will use the interaction of a robotic agent with two WoT Things to illustrate the introduced annotations and mappings. The setup consists of a controller capable of opening and closing an automatic door and an intelligent power supply that provides diferent voltage levels to the door. Specifically, the power supply must provide 5 volts to open the door and -5 volts to close the door. The goal for the robotic agent in the scenario is to open the automatic door. Figure 1 shows the setup.
The power supply has interactions that allow users to read the current voltage output and to change the voltage output by adding a value between -10 and 10 to the current value. A TD of such a power supply following the W3C TD 1.1 standard, is shown in listing 1.
The automatic door controller can read the current status of the door and provide interactions to either open or close the door. But, for the door to open, two conditions must be met: the power supply must be set to output 5 volts and the door must be in the closed state. Conversely, to close the door, the power supply must be at -5 volts and the door must be in the open state. A TD of such an automatic door controller following the W3C TD 1.1 standard, is shown in listing 2.
However, both TDs lack information about the preconditions and efects of interactions. For example, an agent will currently not be able to determine that to open the door, the VoltageOutput property must be 5 volts, or that changeVoltageByValue changes the VoltageOutput property.
Modelling sequential behavior and efects in TDs has been investigated before. Korkan et al. [
Planning algorithms in combination with autonomous robotic agents have been used to
tackle a variety of challenges, such as robot navigation [
The possible combination of TDs and PDDL has been considered before by Noura et al. [
Ontology Requirements Specification Document
only the TD annotated with precondition and efect information to dynamically transform TDs into a PDDL planning problem solvable by standard PDDL planners. Furthermore, our focus is on interactions between robotic agents and Things, rather than between non-professional users and Things.
To describe a mapping between interaction afordances in TDs and PDDL descriptions, additional
semantic annotations for conditions and efects are required. Therefore, we introduce an
ontology called Simple Planning Annotation with a preferred prefix of spa. In the TD we focus
on the basic interaction afordances of reading a property, writing a property, and invoking an
action. We developed spa using the Linked Open Terms (LOT) methodology [
Use case specification The use case specification activity outlines potential use cases of the spa ontology by listing concrete applications. During the creation of the ontology, we identified several potential use cases, which are documented online1. Below, we present the use case representing the running example.
Use Case: An autonomous robot wants to pass through an automatic door.
Description: A robotic agent operates in an environment that includes two IoT devices: an intelligent power supply and an automatic door controller, both controllable by requests. The devices are described by TDs, which turn them into Things. To interact with these Things, the robotic agent must analyze the available interaction afordances and evaluate the preconditions and efects of each interaction to achieve its goal of opening the door.
Actors: robotic agent, automatic door controller, intelligent power supply.
Flow: The robotic agent sets the goal of opening the door, discovers and consumes the TD of the automatic door controller, and finds a link to the TD of the intelligent power supply, which the agent also consumes. Next, the agent evaluates all afordances, associated preconditions, and efects, and maps them to a PDDL planning problem. A PDDL planner then solves the planning problem and returns the sequence of interactions required to achieve the goal.
Functional Ontological Requirements In addition to the general behavior-describing nonfunctional requirements, competency questions are utilized to specify document content-specific requirements. Some sample competency questions extracted from the use case presented are listed in Table 1.
The final ontology requirements specification document is shown in Figure 2.
4.2. Ontology Implementation
The ontology model is described using RDFS vocabulary elements. To reuse definitions and
make the way precondition terms are evaluated unambiguous, we base ontology operations
on operations defined in the W3C’s XPATH recommendation 2. For instance, our ontology’s
spa:NumericMultiply is mapped to XPATH’s op:numeric-multiply. Other mathematical
expressions are based on OpenMath3 definitions, such as spa:LogicalAnd or spa:LogicalOr.
Finally, the qualities of our ontology in terms of structure, semantic representation, and
interoperability were evaluated using the OntOlogy Pitfall Scanner! [
The introduced spa ontology allows us to describe interaction preconditions and efects of the intelligent power supply and the automatic door controller, as can be seen in listings 3 and 4. Listing 3: Excerpt of an intelligent power supply TD showing the precondtion and efect annotations using the spa ontology. 1 "title": "PowerSupply", 2 ... 3 "actions": { 4 "changeVoltageByValue": { 5 "input": {"@id": "changeVoltageInput", "type": "integer", "minimum": -10, "maximum": 10}, "forms": [{ href: "http://ex.org/changeValue" }], "effect": [ { "assign": { "numeric-add": [ { "@id": `outputVoltage` }, { "@id": "changeVoltageInput" } ] }, "to": { "@id": "outputVoltage" } } ] }
A PDDL planning problem consists of a domain description and a problem description [
Mapping Object Types The first step in the mapping process is to identify object types. Object types defined in a PDDL domain description are types that can be used for objects in a PDDL problem statement. When transforming the annotations of a TD into a domain description, we create a Thing type for each discovered Thing. The Thing type allows the assignment of specific predicates, functions, and PDDL actions to a particular Thing, which is necessary in multi-Thing planning domains. 4https://paul.ti.rw.fau.de/~jo00defe/voc/spa# 5https://github.com/wintechis/SimplePlanningAnnotation Listing 4: Excerpt of an automatic door controller TD showing the precondtion and efect annotations using the spa ontology.
Mapping Predicates and Functions Predicates represent boolean values, while functions represent numeric values - specifically, integers or floats - that describe properties of specific objects or the domain itself. Predicates are created for all boolean properties that are exposed by WoT read or write property interactions, as well as for all annotated hidden properties that are not exposed. In addition, a predicate used as flag with the sufix Read is created for all read properties, indicating whether or not a WoT read property interaction was performed on that predicate. Functions are created in a similar way for all numeric properties exposed by WoT read or write property interactions, and for all annotated hidden properties that are not exposed. All generated predicates and functions are associated with the Thing type of the particular Thing.
Mapping Actions PDDL actions consist of parameters, which can be predicates, functions, and objects on which the PDDL action is performed. They have preconditions, which are logical expressions composed of predicates and functions that must be satisfied for the PDDL action to be performed. In addition, an efect describes the changes to predicates and functions after the PDDL action has been successfully executed. WoT read properties are translated into PDDL actions using the precondition annotated in TDs, and have the efect of setting the associated predicate indicating a read to true. WoT write properties and WoT actions are translated into PDDL actions in one of two ways, depending on the data type. If the data type is boolean, two actions are created with the precondition as annotated in the TD and with the efect of setting the associated predicate to either true or false. If the data type is a numeric value, for each available input value and annotated step size one PDDL action is created with the precondition as annotated in the TD and the efect of setting the numeric value function representing the WoT property to the input value.
Domain Description of Running Example The first step in applying the mapping to the running example is to generate two object types: a Thing0 type representing the automatic door controller, and a Thing1 type representing the intelligent power supply. The second step is to generate predicates and functions. For the automatic door controller, we generate two predicates: one indicating if the doorOpenState has been read, and one representing the doorOpenState. For the intelligent power supply, we generate a predicate indicating if the outputVoltage has been read, and a function representing the current outputVoltage value. Finally, we generate the PDDL actions. For the automatic door controller, we generate PDDL actions to open and close the automatic door, and to read the doorOpenState. For the intelligent power supply, there are PDDL actions to change the current outputVoltage for each possible value of changeVoltageInput, and to read the outputVoltage. An excerpt of the generated domain description can be seen in listing 5.
Listing 5: Excerpt of the PDDL domain description based on the running example. 5.2. Problem Description Generation A PDDL problem description contains the concrete objects, the initial state and values of PDDL predicates and functions, and a goal definition.
Mapping Objects Objects represent concrete elements that exist in the problem description and are associated with object types defined in the domain. For each Thing involved, an object with the name defined by the title annotated in the TD is created and associated with the corresponding Thing type.
Mapping Initial State and Values The initial state of predicates and the values of functions can be determined by reading the exposed WoT properties of the associated Things. If properties cannot be read directly, they must either be observed by another device, such as a sensor, or be predefined. We assume that WoT properties are static and do not change after being read.
Mapping the Goal The goal in a PDDL problem description is a logical expression, consisting of predicates and functions, that specify the desired goal state. The planner’s task is to determine a sequence of actions that will transition the system from its initial state to a state that satisfies the goal expression, indicating that a solution to the planning problem has been found. We assume that the goal expression is predefined.
Problem Description of Running Example In the PDDL problem description, the objects must first be instantiated: one object for the DoorController with type Thing0, and one for the PowerSupply with type Thing1. Next we need to set the initial states. The initial values are obtained by invoking WoT readProperty interactions, which results in doorOpenState being false and not read, and outputVoltage being 0 and not read. An excerpt of the generated problem description is depicted in listing 6.
Listing 6: Excerpt of the PDDL problem description based on the running example.
(:init (not (doorOpenState DoorController)) (not (doorOpenState_Read DoorController)) (not (outputVoltage_Read PowerSupply)) (= (outputVoltage PowerSupply) 0) )
We have created a prototype implementation of a Thing Description to PDDL (TD2P) processor6
using the mappings introduced in section 5 in JavaScript. The implementation is based on
the W3C WoT Scripting API reference implementation node-wot7. node-wot handles the
consumption of TDs and the interaction with Things via the WoT abstraction. Additionally the
TD2P processor uses a link following algorithm [
To validate our system, we implemented a simulation that covers the running example with a automatic door controller and an intelligent power supply. The source code of the simulation including a short demo video8 can be found as an example in the GitHub repository of the TD2P processor. The simulation demonstrates that our approach allows the automatic generation of PDDL domain and problem description files usable in standard PDDL planners.
To evaluate the scalability of the system as the number of PDDL actions, and thus the search space, increases, we systematically expanded the number of WoT actions of the simulated devices in the running example. During the evaluation, other parameters such as the goal or the number of devices remained unchanged. We evaluated the TD2P processors runtime and memory consumption to create the PDDL domain and problem descriptions, and the total runtime and memory consumption to solve the planning problem.
The tests were conducted on consumer hardware equipped with an i5-10500H CPU and 16 GB of RAM running Ubuntu 22.04 LTS. Total runtime and peak memory usage were measured separately using the time command in Linux. To solve the planning problem we used the ENHSP-19 planner. All measurements were repeated five times.
]s 106 m [ sed 104 p laE 102 e iTm100 ] BK108 [ ry 106 om104 e M102 k ea 100
P 10
100 1,000 10,000
10
100 1,000 10,000
The resulting diagrams in Figure 3 show that for up to 100 WoT actions, the TD2P processor accounts for the majority of total processing time (between 51% and 67%) and total memory usage (between 96% and 99%). But beyond 1,000 WoT actions, the PDDL planner consumes most resources, and the share of the TD2P processor accounts for a minority of the total processing time (between 2% and 16%) and total memory consumption (between 18% and 29%).
As the number of WoT actions increases from 10 to 10,000 (an increase by a factor of 1,000, or about 99,900%), the runtime for the TD2P processor increases from 420 ms to 480 ms (an approximate increase of 14.3%), and the memory consumption increases from 115,532 KB to 126,400 KB (an approximate increase of 9.4%), indicating vertical scalability for the TD2P processor. 8https://github.com/FreuMi/TD2P/tree/main?tab=readme-ov-file#demo
In this paper, we introduced the spa ontology that is used to annotate preconditions and efects of diferent WoT interactions in a TD. Additionally, we introduced a mapping between the annotations in TDs to PDDL domain and problem definitions, allowing robotic agents that have discovered a Thing to use established AI planning algorithms to dynamically evaluate interactions to achieve a given goal. As next steps, we plan to release the spa ontology as a full OWL ontology with OWL-restricted resources, focus on the correctness of manually scripted WoT programs, and extend the TD2P processor to cover more complex WoT interactions.
This work was funded by the Bayerisches Verbundforschungsprogramm (BayVFP) des Freistaates Bayern through the KIWI project (grant no. DIK0318/03).