Autonomous mobile robots involved in trafic situations with other robots must be able to autonomously adapt their control and perception behavior to the overall trafic situation in those shared environments. This adaptation requires each robot to (1) be able to reconfigure its control and perception algorithms to the current situation around the robot, and (2) have the knowledge about which control and perception algorithms it has available at any instant and select the appropriate ones. This paper introduces the mereo-topology of a skill knowledge graph as the formal representation of the knowledge needed for the above-mentioned online self-adaptation, in addition to the knowledge about how to execute navigation maneuvers. It explains how and why a skill graph must connect to four other knowledge graphs, about tasks, resources, environments and object afordances. The approach is validated in a multi-robot navigation case with dynamic trafic layout inside an indoor corridor.
When involved in trafic situations with others, autonomous mobile robots (AMRs) must be able to adapt their behavior to the overall trafic situation. Robots realize their behavior by executing skills: compositions of software activities providing control, perception, world modeling and decision making functionality that work together to perform a specific task [ 1]. For AMRs, these are navigation tasks such as cruising, crossing a junction, overtaking, or parking. For robotic arms, these are manipulation tasks such as inserting, aligning, or polishing. In addition to these task-realizing activities, skills also run monitoring activities for, among other things, checking continuously if the robot’s hardware and the activity software architecture are behaving correctly, or checking that the currently selected skill is still appropriate.
The functionality of all activities inside the skill must be configured based on the task the skill must perform, the environment in which the it executes, the objects it must deal with, and the resources it has available. In addition, each robot that shares resources and space with other robots must coordinate its own behavior with the other robots to make eficient and deterministic use of these resources and space. That coordination typically has multiple configuration options as well. For example, a threshold on what is a safe closest distance to other AMRs, or a level of assertiveness in navigation maneuvers that involve priority. More often than not, such configuration is done manually at design, implementation or deployment time. However, robots that autonomously adapt their behavior to their situation (that is, robots that are situation aware), should be able to configure their skills themselves at runtime.
This paper presents a methodology to allow robots to configure and coordinate their skills by reasoning on knowledge graphs. Due to size constraints, only the mereo-topological levels [2] of the ontology are discussed. A knowledge graph has entities with a set of properties, connected through edges with the edge label indicating the meaning of the relation between the connected entities. The skill graph is key in the presented approach, because it connects knowledge about the robot’s behavior to knowledge of the task, environment, resources and object afordances.
Two software activities are proposed to support runtime skill (re)configuration: (
The presented methodology is validated using a multi-robot navigation scenario where a corridor area is shared between diferent AMR agents. The corridor always has an active trafic layout which is mediated by a trafic layout mediator, that can suggest trafic layout changes based on the number of robots that want to use the corridor. It is shown that by utilizing the proposed skill graph and activities, a robot is able to autonomously adapt its control behavior to the active trafic layout.
Self-adaptation of robot software has been studied in a number of works in the literature [3]. A common pattern is to divide the robot’s software into a managed and managing subsystem, where the managed subsystem is responsible for the functional behavior of the robot, and the managing subsystem is responsible for monitoring the execution of the managed subsystem and reconfiguring it when needed.
Managing subsystems are often, but not always, implement using a MAPE-K loop [4], which consists
of four diferent steps: (
Previous work presenting models for runtime reconfiguration typically focuses on modeling the components in the robot’s software architecture. TOMASys [5] is a metamodel to represent the software components of a robot and their possible configurations. These models are used at runtime by a managing subsystem, the metacontroller, to reconfigure a robot’s software architecture to recover from faults such as sensor failure.
Likewise, MARTE::ARM-Variability is a UML-based metamodel for representing robot software variability [6]. This model focuses on abstracting the reconfiguration mechanisms supported by the robot software, thus creating a standard interface between the managing and managed subsystems.
The ROSA model presented in [7] represents the robots software components and a set of reconfiguration plans, and is used to support task-and-architecture co-adaptation [8, 9]. This allows the robot to adapt its task plan in case no valid software architecture reconfiguration can be found.
This work departs from a similar model of the skill’s software architecture and connects them to the knowledge of particular situations which the robot can handle, and shows how the robot’s software can be reconfigured when the robot’s situation changes.
Robot skills are compositions of concurrently running software activities for perception, motion control,
monitoring and decision making that work together to realize a specific task. For robots to (re)configure
their skill activities at runtime themselves, they must know how diferent skill configurations are related
to diferent tasks, resources, environments and objects. This section introduces (
The skill software architecture model is shown in Fig. 1a, which defines the software relations between skills, their activities and the activity’s parameters. We define three possible relations between activities and parameters: input parameter, output parameter, and configuration parameter. Input and output parameters represent the input and output data streams that are consumed and produced by an activity, respectively.
Configuration parameters influence the algorithms inside an activity that compute the input-output relations and must be configured before the skill can be executed. The value of configuration parameters depends on the robot’s situation. Therefore, when the robot’s situation changes, through a change in task, environment or resources, then the value of the configuration parameters should be changed accordingly. In contrast, the value of input parameters can change every time the activity is run.
Changing a configuration parameter’s value often requires a coordination to be executed to not disturb the determinism of the input-output relations of the running activities. Input, output, and configuration parameters are modeled as relations, as a parameter can be an input parameter in one activity while it is an output parameter in another.
The skill behavior model, shown in Figure 1b, links a skill to the agents that execute skills, the tasks they perform, the resources they use, the environments they can be used in and the object afordances they act on. The behavior model is formed as a composition of sub-graphs, represented by the nodes in Figure 1b. The agent graph represents agents, which are software entities that autonomously perform assigned tasks by executing skills. Each agent controls a set of software resources (activities), and hardware resources that are either atomic (sensors and actuators) or composed (robot body composed of sensors and actuators). For example, a robot is considered an agent because it autonomously performs tasks by controlling its hardware resources to interact with the environment.
Environments are represented using semantic maps [10], which add semantic labels on top of a geometric map. These labels add additional meaning on top of the base map such as: areas, trafic skill
uses has layouts and objects. Each object in the semantic map has object afordances, which define the actions that can be taken on them by resources. Semantic maps can be dynamic [11], meaning that the map can be updated at runtime based on changes in the environment. During the validation case in Section 4, the semantic map is updated to change the active trafic layout in the corridor.
A task is specified as a set of actions that must be taken on specific objects to reach a goal. The object afordances therefore link the environment, task and resources together.
The behavior skill model is a metamodel in the sense that each of the sub-graphs in the behavior skill model can have diferent instances. Each instance of the sub-graphs represents a model of a particular type of agent, skill, environment, etc. For example, two diferent instances of the environment graph could be a corridor graph and a junction graph, representing the two types of areas.
A concrete instance of the behavior skill model, called a situation graph, defines how a particular software skill model is linked to particular task, resource, environment, agent and object afordance models. A situation graph for the corridor navigation situation for the validation case in Section 4 is shown in Figure 2.
A navigation skill is modeled as in Figure 2a, which defines that this particular skill uses three activities: a lane control, a lidar localization, and a position monitoring activity. Each activity has its own set of input, output and configuration parameters. The skill is usable in corridors and can perform navigation tasks, which define what areas to navigate in order to reach a goal area as represented in Figure 2e. The skill can be used by a robot whose agent graph is shown in Figure 2d.
Beside the navigation skill, the robot controls hardware resources whose model is shown in Figure 2c. These contain a lidar sensor, which can observe wall surfaces and produces point clouds, and a wheel drive, which can navigate lanes and consumes velocity setpoints. The corridor itself is modeled as in
To (re)configure a skill at runtime, two activities are proposed: (
The skill configuration query traverses the graph to find values for the diferent parameters defined in the given skill. The query traverses the edges defined in the situation graph which links the agent’s current situation to the given skill. For example, for the navigation skill in Figure 2a, to find a value for the lane parameter in the lane control activity, the query would traverse from the corridor node to the trafic layout node to the lane node. Once an executable skill configuration is found, it is send to the skill execution activity.
The skill execution activity (re)configures and coordinates the running activities in an agent’s software
architecture to execute a desired skill. There are two reconfigurations which the software architecture
should support: (
Skill execution is preempted whenever there is a change in the task, the resources, or the environment. When this occurs, the skill selection activity first performs a skill configuration query to determine whether the currently executing skill can be reconfigured according to the new situation. If this is the case, the skill execution activity can reconfigure the software architecture. Otherwise, a skill selection query is performed to search for a new skill to execute. If no such skill can be found, task execution should be aborted.
The two presented activities show a form of separation of concerns in the proposed methodology. The skill selection activity only needs to know what skill configuration are valid for a particular agent and situation, without needing to know how those skills are executed in a particular software architecture. The skill execution activity does need to know how a skill should be executed so it can (re)configure and coordinate the activities of a particular software architecture. Therefore the proposed activities consist of a software architecture independent skill selection activity, and a software architecture dependent skill execution activity.
The validation of the proposed methodology is done using the following use case. Two mobile robots must each complete a navigation task which requires them to drive through the same corridor. The corridor is divided into lanes by a trafic layout, which is mediated by an external trafic layout mediator, that can change the active trafic layout based on the number of robots that want to use the corridor.
Initially only a single robot (robot 1) is using the corridor, as shown in Figure 3a. The active trafic layout defines a single lane, allowing robot 1 to use all available space in the corridor. After a period of time, a second robot (robot 2) approaches the corridor, but cannot enter because of the active trafic layout. Therefore, it contacts the trafic layout mediator to request a layout change. The mediator then contacts robot 1, proposing to change to a two lane layout. Robot 1 must then decide whether it can reconfigure its executing skill to the proposed trafic layout. If this is the case, the trafic layout is changed and robot 1 reconfigures its skill, such that robot 2 can enter, as shown in Figure 3b.
A knowledge graph was developed containing an instance of the corridor navigation situation graph, whose sub-graphs were shown in Figure 2. The instance of the agent model of robot 1 is shown in Figure 4c. The agent now has an assigned navigation task, whose graph instance is shown in Figure 4e. The task defines only a single corridor that must be navigated to reach a junction. The instance of the corridor model is shown in Figure 4b. The corridor length has been instantiated with a value, and two trafic layouts are defined in the corridor.
Trafic layout 1 corresponds to the trafic layout in Figure 3a, and defines one lane. Trafic layout 2 corresponds to the trafic layout in Figure 3b, and defines two lanes. Trafic layout 1 is defined as active, corresponding to the situation at the beginning of the demonstration.
The instance of the navigation skill graph is shown in Figure 4a. The configuration parameters in the skill’s activities are now connected to values that come from the other sub-graphs. For example, lane 0, lane 1 and lane 2 in the navigation skill are the ones defined in the corridor instance in Figure 4b. Likewise, the resource graph instance in Figure 4d is now connected to the lanes from the corridor graph.
An implementation of the skill selection, and skill execution activities was made to support the runtime reconfiguration of robot 1’s navigation skill. At the start of the demonstration, robot 1 receives its navigation task and the skill selection activity performs a skill selection query to find what skill it can use to navigate the corridor.
The query returns the navigation skill of Figure 4a. The skill selection activity then performs a skill configuration query to find an executable instance of the navigation skill. At the start of the experiment trafic layout 1 is active, that knowledge is used to query for the lane reference. The resulting query configuration is send to the skill execution activity which configures the robot’s software architecture. This results in the behavior shown in the top panel of Figure 3a.
When robot 1 receives a message from the trafic layout mediator to update to the two lane layout, the skill selection activity is triggered to determine whether a skill configuration can be found for that layout. A skill configuration query is used again, this time looking for a valid configuration for trafic layout 2. The query returns a valid configuration, and robot 1 accepts the trafic layout change proposal.
When the trafic layout is changed, the skill selection activity sends the new configuration to the skill execution activity. This activity reconfigures robot 1’s software architecture. This results in the behavior shown in the top panel of Figure 3b.
navigation_skill performs
This paper proposed to use skill knowledge graphs to support runtime (re)configuration of robot skills, where a skill is the composition of concurrently running software activities that perform a specific task. The major contribution is the mereo-topology [2] of the knowledge representation: the mereology of the relevant terms, and the topology of the structural relations between these terms. The full semantics of all terms and relations is beyond the scope of this paper, and the subject of ongoing work.
The skill software architecture and skill behavior graphs are at the core of the mentioned separation. The skill software architecture graph defines what software activities must run to execute a specific skill, and what parameters must be configured. The skill behavior graph is a metamodel that links the skill representation to sub-graphs of agents, tasks, resources, environments and object afordances.
Instances of the behavior skill graph, called situation graphs, define how a particular skill is linked to models of particular task, agent, environment, resource and object afordance types. Situation graph instances are defined by instantiating the sub-graphs in the situation graph for a concrete application. These sets of graphs contain the relevant knowledge to find configurations of the robot’s activities, which was demonstrated in the validation case.
Two activities, the skill selection activity and skill execution activity, where proposed that query the knowledge base at runtime to find executable skill configurations, and reconfigure a robot’s software architecture based on the selected skill.
A validation case was presented where the trafic layout inside a corridor was changed from a one-lane layout to a two-lane layout based on the number of robots that needed access to the corridor. In the validation case, the presented graphs allowed the skill selection activity to select the right control activity configuration when the trafic layout was changed from a one-lane to a two-lane layout. The skill execution activity was able to reconfigure the robot’s control activity according to the selected configuration.
The advantage of the presented topology is a pragmatically scalable separation of concerns: (
A limitation of the current approach is that the robot can only configure skills in known situations, since it can only query the situation models in its knowledge base. If a situation were to occur for which there is no situation model available, then no skill configuration can be found. However, chances are low that the available knowledge and software activities would sufice to deal with those situations.
Future work will focus on formalizing an ontology based on the presented approach, and on exploring how the methodology and software activities can be extended to support multi-agent skill execution, where diferent agents must coordinate their skill execution for a particular task.
This work was supported by the Flanders Make project Sitanav (Situational Aware Navigation and Mapping), and by the Agribot project with grant agreement id: 101183158.
The author(s) have not employed any Generative AI tools. [5] C. Hernández, J. Bermejo-Alonso, R. Sanz, A self-adaptation framework based on functional knowledge for augmented autonomy in robots, Integrated Computer-Aided Engineering 25 (2018) 157–172. doi:10.3233/ICA-180565, publisher: SAGE Publications. [6] D. Brugali, Modeling variability in self-adapting robotic systems, Robotics and Autonomous
Systems 167 (2023) 104470. [7] G. Rezende Silva, J. Päßler, S. L. Tapia Tarifa, E. B. Johnsen, C. Hernández Corbato, Rosa: a knowledge-based solution for robot self-adaptation, Frontiers in Robotics and AI 12 (2025). [8] V. Braberman, N. D’Ippolito, J. Kramer, D. Sykes, S. Uchitel, An extended description of morph: A reference architecture for configuration and behaviour self-adaptation, in: R. de Lemos, D. Garlan, C. Ghezzi, H. Giese (Eds.), Software Engineering for Self-Adaptive Systems III. Assurances, Springer International Publishing, Cham, 2017, pp. 377–408. [9] J. Cámara, B. Schmerl, D. Garlan, Software architecture and task plan co-adaptation for mobile service robots, in: Proceedings of the IEEE/ACM 15th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS ’20, Association for Computing Machinery, New York, NY, USA, 2020, p. 125–136. URL: https://doi.org/10.1145/3387939.3391591. doi:10.1145/3387939.3391591. [10] A. Nüchter, J. Hertzberg, Towards semantic maps for mobile robots, Robots and Autonomous
Systems 56 (2008) 915–926. [11] S. Van Baelen, G. Peeters, H. Bruyninckx, P. Pilozzi, P. Slaets, Dynamic semantic world models and increased situational awareness for highly automated inland waterway transport, Frontiers in Robotics and AI 8 (2022) 739062:1–25. [12] D. Vanthienen, M. Klotzbuecher, H. Bruyninckx, The 5c-based architectural composition pattern: lessons learned from re-developing the itasc framework for constraint-based robot programming, JOSER: Journal of Software Engineering for Robotics 5 (2014) 17–35.