-Robots should adjust their actions to suit the surrounding situation. This requires robots to be able to interchange signals to symbols and symbols to signals. We propose an intelligent application development platform based on the Service Oriented Architecture (SOA) and Semantic Web Service (SWS). Our platform has five layers: Service, Process, Module, Knowledge and Data. The first three layers are implemented by using OWL-based Web Service Ontology (OWL-S). In the Module layer, there are two kinds of module: modules for action and modules for recognition. By combining these modules, this architecture can handle symbols and signals. In this study, we applied our method to RobotCafe, which is a cafe where robots work as waiters, to enable the robots to change their greetings to suit the situation when customers come and to assist the services of the cafe.
I. INTRODUCTION
Service robots are playing an increasing role in daily life
and various fields. However, due to their restricted functions
and movements, they can only undertake specific services;
one robot cannot easily handle many services. Therefore,
robots should cooperate with each other to provide services.
This cooperation is called Multi-Robot Coordination (MRC),
and we approached this issue by applying SWS for MRC[
However, if robots cannot sense the surrounding situation, they cannot provide good service like a human, even if they cooperate with other robots. So, to make better service robots, robots should be able to integrate symbols and signals in order to adjust their activities dynamically by interchanging symbols to signals and signals to symbols.
In this paper, we propose an intelligent application development platform based on SWS and OWL-S in order to complete a task given by a user by coordinating with other robots. By using our platform, users can create robot services easily by combining multiple software modules. Our platform has five layers: Service, Process, Module, Knowledge and Data. The first three layers are implemented by using OWL-S. Processes of OWL-S have inputs and outputs, so a process can easily be combined with other processes by binding a process’s input to another’s output. In the Module layer, there are two kinds of module: modules for action and modules for recognition. By combining these modules, this architecture can handle symbols and signals. This case study of RobotCafe shows how our proposed platform can be used to easily construct a service that can be changed dynamically to suit the situation.
The ubiquitous network robot (UNR)[
RoboEarth[
C. OWL-S
OWL-S[
These previous works show how easily robot services can be made; our research focuses on this issue.
This section outlines our proposed system architecture. Figure 1 shows the hierarchy of the intelligent application development platform, which consists of five layers.
Service layer: This layer is a task in the real world. Users can make a robot service by using a service or a combination of some services. A task is realized by a sequence of a number of processes according to a workflow.
Process layer: This layer means a subtask composing a task in the real world. Processes are realized by some robots and some sensors. Each process has a flow which consists of some modules. Processes from more than one service can be used as a component that implements the service.
Module layer: This layer is a minimum software module, which means a robot’s activity or a sensor’s recognition. Modules are divided into two kinds: modules for action and modules for recognition. Modules for action are software modules that robots perform. Modules for recognition are software modules for sensors that recognize signal information such as age, facial expression, poses and objects.
Knowledge layer: This layer consists of some
ontologies and rules which are used by modules in
order to acquire knowledge about tasks, circumstances
and so on. For example, in this layer, there are the
Japanese Wikipedia Ontology (JWO)[
Data layer: The data layer contains data used for module processing. For example, the data layer contains environmental maps, instance data, a gesture database and dialog history.
To realize this architecture, OWL-S is used. Modules are
defined as atomic processes in OWL-S, while services and
processes are defined as composite processes. Each service,
process and module has inputs and outputs. Inputs and outputs
correspond to classes of domain ontologies. Users can
combine modules, processes and services with each other by
binding before one’s output to the next one’s input. Some
modules for action use domain rules written in the Semantic
Web Rule Language (SWRL)[
This section explains in detail the proposed architecture based on a case study of the robot service: RobotCafe. In RobotCafe, we use a robot named “NaoTorso”, which has an upper body of the humanoid robot “Nao” and a mobile robot “Turtlebot2”. Figure 2 shows an overview of RobotCafe. RobotCafe has two services: customer relations and tray service. The output of the former service binds to the input of the latter.
Each service has several processes and the output of the previous process binds to the input of the next process. Each process has a sequence of some modules. In this paper, we focus on the process of greetings in customer relations. The greeting process has nine modules as shown in Fig. 3, where rectangles, rounded rectangles and ovals represent the modules comprising the greeting process, arrows mean flows, and labels of arrows mean the input and output of the modules. In the blue rounded rectangles, the signals sensed by cameras are changed to symbols such as age and the kind of object. In a red oval, symbols such as face direction are changed to signals which go forward to the next module.
In the greeting process, a speech robot search is done first
and returns the IP address of the searched robot. According to
this IP address, the robot performs age recognition and object
recognition. In age recognition, the robot estimates the
customer’s age. In the age recognition module, we use the
NAOqi API[
In this flow, the class hierarchy and properties of the RobotCafe ontology shown in Fig. 4 are used as inputs and outputs of modules. Related to the greeting process, the Person class, Object class, Robot class and Cafe class are determined in this ontology. The Person class has subclasses such as Adult class and Child class, which are used to judge the person type of the customer. The Object class has subclasses such as Umbrella class and Baggage class, which are used to determine the greeting comment based on the customer’s belongings.
The Person class has two properties: age property and hasObject property. The age property’s domain is the Person class and its range is int type. The hasObject property’s domain is the Person class and its range is the Object class. The Robot class has the greeting property, which represents the contents of greeting speech. The greeting property’s domain is the Robot class and its range is String type. The Cafe class has two properties: hasClient property and hasRobot property. The hasClient property represents the relationship between the cafe and the client, while the hasRobot property represents the relationship between the cafe and the robot which is in the cafe. The hasClient property’s domain is the Cafe class and its range is the Person class. The hasRobot property’s domain is the Cafe class and its range is the Robot class.
Figure 5 shows a part of the SWRL rules for RobotCafe. In Fig. 5, there are rules for judging the person type based on age and rules for judging the greeting speech contents. For example, in the former rules, there are rules that a person has the YoungChild class when the person is 12 years old or younger, and has the TeenAge class when the person is 13 to 19 years old. In the latter rules, there are rules such as that a robot says “Hello. Thank you for coming.” when the customer belongs to the TeenAge class or Adult class and that a robot says “Hi. Where are your parents?” when the customer belongs to the Child class.
In addition, there are rules related to the customer’s belongings in rules for judging the greeting speech contents. For instance, there are the rules that a robot says “Would you mind putting your umbrella there?” in addition to the usual comments when the customer has an umbrella, and rules that a robot says “Could I take care of your baggage?” in addition to the usual comments when the customer has baggage. Therefore, by using these rules, the greeting comments can be tailored to the situation such as the customer’s age and belongings.
Table 1 shows the types, names and details of inputs and outputs of modules composing the greeting process. In this table, modules whose input or output class is null such as the IP (Internet Protocol) address of the robot and recognized age have no RDF graph input or output but basic data type input or output. Figure 6 shows the detailed processing of the acquisition by the greeting contents module. This module inputs RDF graphs merging the Person model, Robot model and Cafe model. In this module, rules for judging the greeting speech contents are used as inputs. Then, this process makes a triple which defines the greeting contents by using the greeting property and outputs the triple. Figure 6 shows an example in which the greeting content is “Hello. Thank you for coming. Would you mind putting your umbrella there?” for a customer who belongs to the MiddleAge class and has an umbrella.
As mentioned above, RobotCafe can be implemented by using the proposed architecture based on modules, processes, services, ontologies and rules and can be dynamically changed according to the circumstances.
As an example of the robot service using symbols and signals, we describe the greeting process in RobotCafe. In this process, we use a robot called “NaoTorso”, which is a combination of a humanoid and the mobile robot “NaoTorso with Kobuki”. This robot has an upper body of the humanoid robot “Nao” and a mobile robot “Turtlebot2”. This robot is good at dialogue and stable movement.
Figure 7 shows the greeting process. When a customer comes, NaoTorso approaches the customer and speaks the greeting dialogue.
Figure 8 shows the implementation of person detection.
The left side of Fig. 8 shows an example of person detection,
which is implemented by using Choregraphe[
The lower right side of Fig. 8 shows the appearance of the person and the robot.
The greeting process is performed as follows, which shows the greeting comments spoken by NaoTorso according to the customer’s situation. A child (whether with belongings or not)
An adult without belongings
An adult with an umbrella
NaoTorso: Hello! Thank you for coming. Would you mind putting your umbrella there? An adult with some baggage
NaoTorso: Hello! Thank you for coming. Could I take care of your baggage?
In this paper, we proposed an intelligent application development platform based on SWS and OWL-S. This architecture enables users to integrate symbols and signals and to make robot services easily which can be tailored to the circumstances. The architecture has five layers: Service, Process, Module, Knowledge and Data. The upper layer consists of the combination of components in the lower layer. In the case study, we showed a greeting service in a cafe which can be changed according to the customer’s situation.
In the future, we will create many modules for actions and modules for recognition in order to achieve different and difficult tasks. Then, we will create modules for affect/emotion processing. In addition, we will build the architecture in order to manage robots and sensors and a workflow editor which enables users to use this intelligent application development platform easily. Finally, we will do more testing in complex scenarios involving many processes.