This paper aims at describing, from an industrial perspective, the experience in delivering conversational agents via the development of Iride, a platform able to deploy multi-language task-oriented dialog systems. It has been implemented a set of functionalities that can be aggregated in different ways, in order to build domain independent conversational systems, which are able to satisfy needs of real business cases. Along with algorithms and techniques for end to end Dialog management, such as Natural Language Understanding (NLU), Question Answering (QA) and Dialog State tracking and policy management, the technical insights leveraged into the platform are described by outlining the requirements and constraints emerging from these on the field experiences.1
Over the last years the human computer conversation has been gathering increasing attention due to its promising potentials by opening up a new profits-making market segment. 2. The benefits of using dialog systems are manifold, these systems can answer to complex questions and also handle hundreds, thousands of conversations at the same time, reducing response times and probability of error in repetitive tasks. In General, developing conversational agents at industrial level requires to manage several issues: (i) The lack of real data: in the majority of the real business 1Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
2https://www.gartner.com/smarterwithgartner/4-trendsgartner-hype-cycle-customer-service-customer-engagement/ cases, in our experience, not enough data are available for training pure learning methods, moreover, the research datasets do not fit the industrial purposes; (ii) Domain updates and system maintenance: The domain requires continuous updates (e.g. the introduction of a new product or service) and the delivered system needs the maintenance, update or changes to correct faults and to improve performance; (iii) User Experience: the conversational agent is the front end of the company, multi-modality (i.e. different user experiences depending on different devices) and what the company aims at communicating must be taken into account; (iv) Runtime latency: is required to add no more than few mini seconds to the entire serving stack; (v) Scale and quality of the text collection: in a voice interaction the system cannot answer with a long text document, but needs to answer with a clear short document passage; (vi) Certified Answers: Being the virtual assistant the voice of the company, it must be controlled (i.e. usually the answers and the messages communicated by the assistant have to be certified by the company); (vii) Human in the loop: Although virtual assistants are becoming more and more intelligent, they are not able to satisfy every user need. In this scenario, it would be better a mixed management, combining the use of virtual agent and human operator.
In this paper, we describe the Almawave’s developed solution that allows us to quickly design, write and deploy interactive conversational systems without coding, enabling non-technical users (i.e. conversational designers or domain experts) to design conversational agents, and it leverages Natural Learning Processing (NLP) and Machine Learning (ML) to develop a human-like experience for users. This framework is designed to build multi-turn task-oriented dialog able to solve defined tasks and answer to domain questions.
Following, in section 2 related works will be discussed: in section 3, the various goals that have leaded the described solution will be discussed; in section 4 the various modules of the architecture will be fully described; finally, in section 5, we formulate some considerations and lessons learned in the conversational agent field. 2
Due to the complexity of task, most studies on
Human Machine conversation have addressed
individual components such as Intent/Slots
detection
From our experience, the main objectives of a dialog system for business needs are the usability and the robustness. The system must always be functioning in time and satisfy user needs by operating as few interactions as possible. The conversational platform here proposed was developed with some characteristics concerning those objectives: 3.1
The usability principles for this kind of framework look to user designers. The main issues to purse the usability goal are described in the following:
Designing a dialog conversation must take into
account both what has already been said and what
will happen next; it is much more complex than
one-off activities, like answering a search query,
playing a song and so on. In relation to this, new
professions are emerging, such as the Voice User
Interface (VUI) designer who curates the
conversation, defining the flow and its underlying logic
in a detailed design specification that represents
the complete user experience, playing an
important role from the conceptual phases of the project
until its launch
However, these profiles are not necessarily developers or data scientist, so it is very important that tools offer to them all the available technology but are easy to use, so that the designer can focus on aspects more related to domain and policies of dialog management. A solution we delivered to solve this problem is a Visual Dialog Editor, hiding the complexity of programming AI components, allowing the user to construct a dialog agent with a visual building block approach, the drawn flow is thus compiled producing the dialog agent software. In Figure 1 an example of a conversational design process is shown, from a conversational map that highlights the important items to the dialog model drawn with the Dialog Editor. possible user input is difficult, therefore the platform provides different solutions to improve the reliability managing both not understood and misunderstood inputs.
A Component Based approach in a SW architecture lead to quality products, rapid development and an increased ability to adapt to change. In contrast to use of end-to-end conversational model that concentrates all the interaction features and capabilities within a monolithic model as blackbox, a modular approach allows the potential of system engineering to be exploited for complexity management. An important aspect we focused was to maximize the re-usability of the platform components, such as algorithms or trained models as well as the dialog flows, within conversational agents for different domains, tasks and languages, maintaining a domain and task independent environment. To pursue it, the framework makes various components and algorithms available, in order to have a different level offer views. There are components dedicated to knowledge management, others that realize language understanding, dialog management and multi-modality connection. Even a single module can be seen as the set of submodules that realize more specific functionalities. 3.2
In a commercial solution the robustness of a system must be guaranteed, and it can be achieved by a combination of different strategies. A significant effort was made in the system to detect and handle a wide range of errors, ranging from the language understanding, the discourse processing and the domain reasoning. But, whatever input understanding strategy is adopted, managing every 4
This section describes the overall structure of the platform. In order to pursue the main goals we defined this architecture. It is the result of collaborative effort between working on the different technologies and where the different components can be assembled to produce multiple applications.
The components are described dividing them in 3 logical views: The Design Tools for the conversational agent design, the Dialog Core Modules that implements the underline engine dialog components and, in order to provide analysis over the conversations, an Analytics Module. 4.1
Modeling a dialog means defining the flow of the conversation and its underlying logic.
Designers define the behaviours of the agent, defining the dialog script in terms of States, Transitions and Actions. The visual editor facilitates the modeling of the flow of dialog, drawing the transitions between the dialog states and actions using graphical approach, and enable the use of the various types of resources.
Moreover, the editor, provides a graphical interface to the resource management (e.g. ontologies, models, indexes).
A conversation simulation environment is provided within the editor for the dialog assessment. This tool enables the testing by the designer and confirms the correctness of the dialog before deployment.
Through the simulator it is possible to verify some relevant aspects in the realization of virtual assistant. Observing the flow of conversation makes it possible to assess the smoothness and naturalness of the discourse, in relation to the management of waiting times and turn-taking. The simulator also helps to evaluate and balance the use of graphic components such as images, buttons and quick replies, usually used to make the interaction easier. It is also important explore the error management to put in way out and recovery policies. 4.2
Designers use knowledge representation to build the operational structure of the dialog agent. The concepts of the domain and their relationships are represented by ontologies, taxonomies and dictionaries. If we could develop a dialog agent in a new domain with a rich ontological structure, re-using the knowledge of the existing domain becomes fundamental. The separation of domain knowledge also reduces the complexity of the linguistic components, using both general purpose resources and domain specific ones. Within the conversational platform different types, i.e, dictionaries, ontologies, inference rules, indexes and machine learning models, of knowledge representation are used in combination in order to obtain flexible dialog and dialog agent configurable.
The platform makes available a proprietary multi-lingual NLP pipeline, composed by several modules that enable language comprehension, providing the language analysis at several levels ranging from morphological to pragmatic and task-dependent analysis.
This pipeline allows an hybrid approach, rulebased and machine learning, depending on needs, that can be both used and combined together, exploiting, for example, the outomes of DL classification into ontological reasoning. Among the several modules, the following Deep Learning models are leveraged:
A sentence classification model built over
pre-trained language models
The chosen models benefit from the advantages
of the transfer learning techniques
The framework allows the use of domain dictionaries, ontologies and inferential rules that enable the extraction and inference of semantic concepts. Our framework gains the benefits of each approach by simultaneously applying the rule-based and machine learning approaches combining both techniques to infer complex knowledge structures. It is worth mentioning that with the platform is released a tool that allows, in a simple way, even to non-technical users the training of specific models to customize a system on a given domain.
The dialog manager (DM) is the core component of the platform. At each turn in the conversation, the dialog management component takes the current dialog state and the user utterance as its inputs, performs different actions based on context, and outputs corresponding results as responses. DM includes two stages: dialog state tracking and dialog policy. The dialog state comprises all that is used when the system makes its decision about what is the next agent action; in this scenario, the dialog state tracker updates the context based on the result of the analysis of the last received input, e.g. NLU analysis over the user utterance or the query response of an external knowledge base.
In the proposed approach, the dialog tracking
is implemented over hand written probabilistic
rules in line with
Question Answering: The Question
Answering action follows two steps: it performs
a retrieval process over a domain dependent
index. The retrieved answers are re-ranked
applying NN for learning to Re-Rank process
as in the CQA task
able for integration with external systems. The conversational designer can graphically draw this action fulfill few input data (e.g. endpoint, authentication and request data)
with the user a set of interactions to fulfill the values of a specified list of entities, e.g. the slot list of an intent or the properties of an ontological concept.
Route to Operator: Under specific conditions, the dialog session can be redirect to a human operator giving to him the visibility of the information acquired up to that time. This action manages specific business cases ensuring robustness and service continuity.
There are different ways of communication and the choice of the users depends on various factors. The platform makes available connectors to different communication channels ranging from social network to legacy systems. The conversational agents can be delivered both through voice and written chat. Moreover, the change of channel is available (e.g. route the chat to operator or vice versa) in order to respond to specific business cases managing the change transparently to the user. Moreover, the conversations based on the different channels, can be equipped with UI components such as images, buttons and quick replies. 4.3
The analysis of the conversations provides a constant view of how the conversational agent plays the "voice of the company" role. The analytics module allows to extract several insights from the dialog: interaction satisfaction, dialog errors as well as analytics for CX analysis. This one, in addition to provide market information, collects data for the agent maintenance and updates.
In this paper we described the experience in building the Iride conversational platform for the design and deployment of task-oriented conversational agents in enterprise environment. The platform has been built taking into account needs and constraints required by an industrial scenario. We focused on a component based architecture able to maximize the re-usability of the components, enforcing a clear separation between the domain-specific aspects of the dialog and domainindependent ones across the several dialog layers (language understanding, dialog management and knowledge management). Moreover, in order to enable the work of conversational designer, the platform offers a suite of tools for conversational designers. Such architectural choices have been verified testing "on the field" the effectiveness and usability of the described solution.
Several conversational agents have been developed with this framework, in different business cases and in different domains and languages; these experiences demonstrate that the platform is efficient and easy-to-use and meets the needs of various types of use cases.