This work presents the design and the implementation of CAIR: a cloud system for knowledge-based interaction devised for Social Robots and other conversational agents. The system is structured in a way that it can be easily expanded by adding new services that improve the capabilities of the clients connected to the Cloud. Another key feature of the system is that it has been designed to make the development of its clients straightforward: in this way, multiple devices (e.g., robots, computers, smartphones, etc.) can be easily endowed with the capability of autonomously interacting with the user, understanding when to perform specific actions, and exploiting all the information provided by services on the Cloud.
CARESSES1 is an international, multidisciplinary project whose goal is to design the first robots
that can assist older people and adapt to the culture of the individual they are taking care of
[
Recently, it has become more and more common to exploit cloud technologies to improve
the eficiency of many devices and systems. In the robotics field, this practice is defined as cloud
robotics, i.e. the use of remote computing resources to enable greater memory, computational
power, collective learning and inter-connectivity for robotics applications [
This paper describes the cloud system that has been designed based on the underlying principles of the CARESSES system, but completely modifying its architecture to ofer a set of web services for multiple robots and devices.
The CAIR system has been developed by taking advantage of the rich knowledge base and
the dialogue mechanism developed during the CARESSES project, intending to create a system
much easier to use and able to manage contemporary connections from multiple users. The
system is based on the use of REST APIs [
The system for knowledge-based autonomous interaction described in this work can be easily used by most devices with Internet connectivity, able to acquire an input through a keyboard or a microphone, and provide an output through a screen or a speaker (e.g., robots, computers, smartphones, smartwatches, etc.).
The system is based on a client-server architecture and it has been designed in a way that it is easy to add new functionalities to improve its performance, and it is easy to be used by the clients.
Figure 1 depicts the proposed architecture. The server is composed of two web services developed in Python: (1) the Dialogue Manager service that manages the dialogue and analyzes the user sentence to recognize the intention of talking about a specific topic, and (2) the Plan Manager service that recognizes the intention of the user to make the agent execute a specific action. To provide appropriate answers and plans, the server exploits an Ontology containing all the topics, keywords, sentences and plans used during the interaction with the user. The Flask-RESTful2 framework has been used when developing the web services.
The client can perform requests to the server using REST APIs. REST is a set of rules that should be followed when creating the API. One of these rules states that the client should be able to get a piece of data (called a resource) when linked to a specific URI. Each URI is called a request, while the data sent back to the user is called a response. Any web service that obeys the REST constraints is informally described as RESTful. Due to the separation between client and server, this protocol makes it easy for developments across the various areas of a project to take place independently. In addition, the REST API adapts at all times to the working syntax and platform. This ofers the opportunity to try several environments while developing.
The client acquires the user sentence, sends it to the server, parses the response, executes the received Plan and/or replies with the dialogue sentence returned by the Dialogue Manager.
2https://flask-restful.readthedocs.io/en/latest/
The Dialogue Manager service is in charge of managing the dialogue. To provide the appropriate
response to the user, the Dialogue Manager service requires as input the user sentence and the
client state. As mentioned before, the ability of the system to naturally converse with the user
has been achieved by creating a framework for cultural knowledge representation that relies on
an Ontology [
Based on the Dialogue Tree, the key ideas for knowledge-driven conversation can be briefly
summarized as follows (the whole process [
Each time a user sentence is acquired: 1. A Dialogue Management algorithm (either keyword-based or based on more advanced topic classification techniques) is applied to check if the user’s sentence may trigger one of the topics in the DT by jumping to the corresponding node; 2. If no topics are triggered, the conversation follows one of the branches of the DT (according to policies that take into account the user’s cultural background and personal preferences).
The system continues in this way, proposing sentences corresponding to a node and acquiring the user’s feedback that can be used to update the user’s preferences and/or determine the next node to move to.
As shown in Figure 1, other than providing the dialogue reply and the updated client state, the Dialogue Manager service also calls the Plan Manager service, working as an intermediary between the client and such service. However, the client could directly call this service if not interested in managing the dialogue.
The Plan Manager service receives as input the user sentence. Its purpose is to find a match between such sentence and one of the trigger sentences of a specific Intent. An Intent is defined by (i) a set of trigger sentences (built using regexes), (ii) one or more plan-specific sentences (if any), (iii) a KBPlan (if any), (iv) and a Plan (if any) (Figure 2).
Trigger sentences (i) are used to check if the user sentence matches with the corresponding Intent. Such sentences are currently modeled with regexes, and allow to extract parameters from the matched sentence that can be used to dynamically compose the plan sentences, the plan and the KBPlan. Besides trigger sentences, also plan-specific sentences have been defined (ii), which are used by the system to reply when an intent is detected.
A KBPlan, where KB stands for Knowledge Base, is a sequence of actions meant to afect the knowledge base and/or the flow of the dialogue. For instance, if the user says “I love music", this sentence will match the trigger sentences of the Appreciation Intent (Figure 2) meant to recognize the user’s appreciation for something and extract the loved thing as a parameter. The KBPlan of this Intent is composed of two actions: the first one increases the probability that the user wants to talk about the extracted parameter, while the second one brings the information that the system should jump to that conversation topic (if present in the Ontology).
A Plan is a sequence of actions that should be executed on the client as it does not afect the knowledge base nor the flow of the dialogue. For instance, if the user says “Play the song Yesterday", this sentence will match one of the trigger sentences of the Music Intent (Figure 2) that recognizes the user intention to some music. The plan of this Intent is composed of a single action carrying the information that the client should play the song having the title contained in the parameter field (see Section 2.3).
If the Plan Manager service finds a match with an Intent, a response containing the KBPlan, the plan sentence, and the plan is returned to the Dialogue Manager service (see Figure 1). The KBPlan is managed by the Dialogue Manager, as it directly afects the continuation of the dialogue, while the plan sentence and the plan are not used by this service. Once the Dialogue Manager service has chosen the most appropriate answer for the user based on the information contained in the client state, the previous user sentence, and, eventually, the KBPlan, it returns a response to the client. Such a response contains the updated client state, the plan sentence, the plan, and the dialogue sentence (i.e., the actual continuation of the dialogue).
A client for CAIR can be easily developed for most devices with Internet connectivity such as robots, computers, smartphones, smartwatches, etc. The first thing that the client should do is to perform a PUT request to the server, in particular to the Dialogue Manager service, to obtain the initial client state and the first sentence of the conversation. The client state should be stored locally and retrieved before all the following requests. Afterwards, the client should acquire the user input, send it to the server along with the client state with a GET request, retrieve and manage the response. This last operation expects the system to communicate the Intent reply to the user, perform the actions contained in the plan field of the response, and eventually continue the dialogue by communicating the sentence reply. If the client is not able to execute the actions it can ignore them and consider only the sentence reply.
An example of a simple client for PC can be found in this GitHub repository3. The repository contains also a PDF guide with a detailed explanation of the code and of all Plans that the server can return to the client based on the Intent that has been matched. Another documented example of a full client for the SoftBank Robotics robots Pepper and Nao, that manages all the Plans returned by the server, can be found here4. A video showing some extracts of the interaction is available on YouTube5.
3https://github.com/lucregrassi/CAIRclient_instructions.git 4https://github.com/lucregrassi/CAIRclient/tree/develop 5https://www.youtube.com/watch?v=UdbYGytd07w