We have developed PRactical INTElligent aPplicationS (PRINTEPS) which is a platform for developing comprehensive intelligence applications. This paper introduces an application of PRINTEPS for customer reception service in robot cafe by using stream reasoning and Robot Operating System (ROS) based on PRINTEPS, and for integrating image sensing with knowledge processing. Based on this platform, we demonstrate that the behaviors of a robot in a robot cafe can be modified by changing the applicable rule sets.
Designing machine-human task collaboration often requires integration of the image sensing technologies that help recognize surrounding circumstances by using a rule set of a target operation. However, a major hurdle exists in connecting the two directly. This is because a huge grain-size difference exists between information acquired through image sensing and that expressed by a rule set.
As a means to achieve integration between rules and image sensing, we
propose a novel method to integrate signals (dynamic sensing data) and symbols
(RDF stream) via Robot Operating System (ROS) [
We conducted a case study of a robot cafe customer-reception service using Pepper 2, which is an emotion-recognizing humanoid robot. As a result, a robot
2 https://www.aldebaran.com/en/cool-robots/pepper Kinect v2 Windows Sensing at entrance
Robot!Pepper" …
Images of faces of people
Age and gender
estimation Entrance Information Event Detection
ROS Environment
Entry detection Stream
Reasoning Speech Dialog Robot speech Robot moving Movement
Multiple Knowledge
Cafe Ontology IF … THEN … Business Rules can use the rule sets that the person has, and we can construct a more efficient robot service system. A demo movie for the customer reception service can be seen on YouTube 3. 2
The customer reception service mainly consists of the “customer detection” process which detects an incoming customer by means of the Kinect sensor and the “greeting to the customer” process which orders the robot Pepper to give the customer a greeting based on the business rules described by semantic web rule langugage (SWRL).
Title Suppressed Due to Excessive Length e #!W)9'c*'U-'_8Z$X#'V8Z#VUK5-.C641OY' >'UK5-.C641f./f16.2C641 dd'O'>'+W)X&VU-YY (detected within 3 s) and Kinect. The distances between the faces of the people detected within 3 s (ID: c1) and Kinect are chronologically shown as c1d1, c1d2, c1d3, and the values of c1d1-c1d2, c1d1-c1d3, and c1d2-c1d3, are computed in order to count the number of values greater than 0.1. The value 0.1 is determined based on the accident error of Kinect’s depth sensor value. This measurement is used to avoid erroneously detecting someone who is in front of the cafe but is not approaching it as a customer. A count exceeding 1 means that someone is approaching the cafe (Kinect), which is a measurement used to avoid erroneously reacting with a customer who is leaving the cafe. The average value of the distances between the faces of the people detected within 3 s and Kinect is also calculated, and if the average value is less than the ?distance to entrance (3.3 m), the customers are recognized as having entered the cafe, and their IDs, the aforementioned count, and the average values are returned. The value of ?distance to entrance is obtained using a SPARQL query for measuring a distance between Kinect and an entrance by predefining, based on the cafe ontology, the distance between the location in which Kinect is installed and the entrance of the cafe. Currently, a distance between a person’s face and Kinect is the only information that is used. However, if the sensor can obtain various attribute information from a person in the future, more complex customer detection based on such information (e.g., discerning a customer from a cafe clerk based on clothing) will be realized. 3.2
Greeting to the Customer Process means that a group of two people is found at the entrance; Pepper is positioned at the entrance; when Pepper provides a service to the group, Pepper gives the statement: “Welcome. Two seats?”
If a cafe owner wants a robot to give a more detailed greeting statement based on a customer’s age, gender, members of the party (e.g., family members, girl or boyfriend), the owner must only revise the rule defined there. 4
In the future, we will try to make more types of information available by using stream reasoning for more image sensing processes to make the services more effective, and to enable easier correction of the system, such as by clarifying how many processes are needed for actually changing the service.
We are grateful to Mr. Yusuke Nakayama of the Graduate School of Science and Technology of Keio University for his software-implementation for this study.
This study has been supported by “A Framework PRINTEPS to Develop Practical Artificial Intelligence” of CREST of JST.