Chatbots are computer programmes that mimic human conversation to interact with users through a variety of messaging channels. They are now regularly deployed on e-commerce and business websites providing customer support. Chatbots have also been employed for research and clinical support in the healthcare domain. In the eld of psychology, chatbots have been applied to clinical research where survey or interview data collection are substituted with chatbots that can interact with the subjects via phone messaging apps in a non-clinical setting. This paper examines the design and development of a chatbot for a clinical psychology research study. The stakeholders, functionality, perspectives and technical challenges are presented and discussed. We apply a quality of experience framework to explore the factors that impact stakeholders and in uence design priories. We present our conclusions regarding the leveraging cloud platforms and the technical customisation required for non-standard chatbot use cases.
A conversational chatbot is a computer programme that tries to mimic a
conversation with a real person. State-of-the-art chatbots rely on underlying Arti cial
Intelligence (AI) and Natural Language Processing (NLP) technologies to deliver
a natural conversation experience [
Conversational chatbots rely on NLP to extract information from the users' text and then react with a highly matched response. Many also use AI to improve the accuracy of the response over time, e.g. virtual agent chatbots.
From a user experience perspective, a general conversational chatbot that
naturally handle various scenarios is desirable. In practice, however, chatbots are
constrained in their design to speci c purposes [
The statistical-based methods use application-speci c text corpora to train
statistical Machine Learning (ML) based NLP systems. These methods can
identify and parse language structures, grammar and phrases to interpret sentences.
The rule-based methods, on the other hand, use a prede ned set of rules such
as WordNet (electronic lexical database for English [
Chatbots have been widely adopted for business applications such as web shopping helpers, hotel reservation agents, and FAQ agents. The applications of these chatbots share a general conceptual design with chatbots designed for the healthcare domain including common stakeholders and the use of NLP. In practice, chatbots have four key stakeholders as de ned in Table 1 along with domain-speci c examples.
In this research, we re ect on the Quality of Experience (QoE) for the
stakeholders involved with chatbots. QoE is de ned as the degree of delight
or annoyance of the user of an application or service [
From a QoE perspective, we can de ne the QoE for each stakeholder of a healthcare chatbot as follows: { End user: The satisfaction level for the patient (study participant) will be proportional to how the chatbot responses to irrelevant inputs by users. User experience (UX) also a ects the overall QoE for the end-user such as the use of buttons and dropdown menus. QoE is a ected by the ease of engagement as well, e.g using Messenger rather than a custom app to be downloaded separately. { Psychologist: The chatbot will result in the same information being collected as would result from questionnaires/ clinical interviews but with less e ort and a more natural environment for subjects to provide more reliable answers and engagement. All results are captured and stored electronically so they can be analysed and studied easily. { Developer: The ease of deployment and integration to multiple messaging channels while maintaining the desired level of functionality. { Agent: The graduate student role has changed from clinical data capture via questionnaire to software development. Their QoE will result from the ease of system development and data presentation. 1.1
Developing of a chatbot platform requires software design, programming skills
and knowledge of related elds such as NLP, ML and AI. However, several
chatbot cloud platforms have been created in recent years, such as Google
dialog ow [
A typical user's message would be: If I want good friendship, I should be honest, in such a case, the chatbot should be able to do a couple of things, rst: what is the best reply? i.e. identifying the user's intention, second: is there any keyword? in this example, ``honest'' is a value that we want to know, and third: do we need to keep any or all of the information for further processing? Using the underlying technologies, these platforms perform these steps for developers.
Moreover, these platforms allow chatbot designers to focus on the tasks and interactions for businesses. However, the platforms target applications where the chatbot conversations are task-oriented, e.g. FAQ, and expect the user to ask a speci c question, e.g. What is the weather forecast today?. This paper examines how such platforms can be used to develop chatbots for a non-task oriented conversation in the healthcare domain.
This paper re ects on the authors' experience while implementing a chatbot using these platforms, the paper does not investigate thoroughly these platforms nor conduct a comparison among them. 1.2
Chatbots are being deployed for various research purposes. H. Lo and C. Lee [
The use of chatbots for psychology purposes has been an active area of
study [
Robert Morris et al. [
Other researchers are also exploring the application of chatbots to
nontraditional use cases, but without a signi cant focus on the issues surrounding
chatbot design and the collections of feedback from chatbot users. Zhou et al. [
As discussed earlier, chatbots can generally be classed as task-oriented or
conversational. However hybrid chatbots such that it is task-oriented with some
small talk conversational capabilities can also be created to ful l a speci c
purpose [
This research focuses on the design challenges faced while developing a chatbot for healthcare purposes. The use case was to design and implement an intervention chatbot for psychological studies, named Plybot. Plybot is an intervention informed by Relational Frame Theory which seeks to reduce instances of problematic rule-following. Speci cally, Plybot targets generalized pliancy wherein people adhere to rules just because they believe they should. Through conversing with Plybot, users examine why they follow their rules and whether or not these rules are useful for them. This paper is motivated by our need to examine the design factors in developing a psychology chatbot that follows a certain conversational intervention dialogue. We looked into the problem from the QoE perspective of chatbot developers as well as the other chatbot application stakeholders. Plybot, the chatbot used as the case study, was developed in collaborations with researchers at UCD School of Psychology to conduct a clinical research study. The study aims to understand the potential for a chatbot to substitute routine visits to a psychologist while maintaining the same level of integrity provided to the patients. Ethical and clinical issues will be discussed in future publications.
In this paper, we discuss the design, implementation and challenges faced in delivering a functioning psychological self-help chatbot. Unlike previously implemented CBT chatbots such as woebot, this paper discusses the technical and design perspective. We focus on the design considerations and development challenges in implementing a chatbot using a cloud platform where the NLP and ML subsystems are abstracted from the developer. 2
This section describes the architecture of the \Plybot" chatbot and discusses the important decisions addressed. 2.1
The Plybot design requirements can be views in contrast to a business-oriented chatbot such as an online shopping helper bot. Table 2 presented the main features required by Plybot and a shopping helper chatbot.
This table highlights the technical challenges to be addressed in developing Plybot compared to a simple FAQ-style chatbot that helps people with online shopping queries. Plybot requires the use of previous data into new conversations. It should also have the capability to initiate the conversations (a mandatory requirement). Unlike task-oriented chatbots, plybot should be able to continue the conversation from where it stopped, therefore a persistent user state should be kept even within the same daily session. These requirements make healthcare chatbots in general more demanding than business-oriented chatbots. 2.2
A cloud-based solution such as dialog ow, Microsoft Bot Framework and IBM Watson are cloud-based chatbot platforms. They provide some advantages over custom-built software solutions, namely: { Customizable dialogue: simple user interfaces to create and to modify the dialogue { Logging: access to data in an easy way for analysis by non-technical users { Security and Ethics: privacy of individual users where the data is protected by the policies of the service provider { Cost: low volume expected, low budget wanted. Cloud platforms pricing models target high volume solutions and are economically attractive for low volume applications. { Scalability: the system can scale on-demand, and grow with the need. { Ease of use/reuse: the implementation can adapt and redeployed for future activities
However, these systems bring their challenges, for instance, unlike Microsoft Bot Framework, dialog ow does not support proactive messages (a message to be sent to initiate a conversation or continue a dialogue with a user after a signi cant pause, e.g. a day later). Yet, dialog ow o ers the majority key features required for a chatbot. One of the most important features of cloud-based systems is the easiness of integration with messaging platforms such as Skype and Messenger. Dialog ow allows integrating the communications channels conveniently and exibly. For this case study, we chose dialog ow having also explored and experimented with the capabilities of Microsoft Bot Framework. We did not investigate the use of IBM Watson conversation for this case study. 2.3
{ Intent: is what the idea or message that the users want to convey to the chatbot. Based on interpreting a user input the chatbot sends a reply or performs an action. For instance, a user may indicate a wish to pause a conversation and continue again at a de ned time. In such an instance, the chatbot should reply with a con rmation if the time is valid, or ask for valid input otherwise.
NLP
Send reply
No Fulfillment? Yes Process the request No
As discussed in Section 2.1, one of the main features required by Plybot is the easiness of interacting with the chatbot application. The requirement was that
e sag Would you like to se hear a joke? M y l p e 'rrs Yeah e s U
Yay! So… why did lyAthe Chatbot cross the pe road? R y l p e r s ' r e s
U ge So we use rules… a ss and sometimes they eM work for us. Right?
lyp of my rules could be e
R users barriers to engagement should be minimised, speci cally, no need for new app installation on user phones. Skype, Messenger and Slack were the three top choices considered for the chatbot. Slack and Messenger have rich APIs compared to that of Skype. Messenger is widely used in everyday life, unlike Slack which is more of a business-oriented messaging platform. Moreover, healthcare chatbot applications require privacy and security level not provided via Slack as it is based on channels where each user can see all other users. For data privacy, Skype does not send any information about the user id with message replies, making data logging and retrieval more challenging. However, there is a unique part of the id eld for every user, e.g., "id": "4074b0s6-5871-4638-907c-80fcfe8461b3-283dg5df" . The last 8 digits are unique for each user and do not change over from day to day. On the other hand, using Messenger is more straightforward as the Facebook Page ID (PID), which is unique for every user, is usually sent with every message, simplifying the implementation when using the Messenger API. This PID is labelled as "facebook sender id": "4050067363115363" which is used to communicate directly with the user using Messenger API and access credentials. For these reasons, Facebook Messenger was selected as the messenger platform for this case study. 3.2
Figure 3 shows the sequence diagram for Plybot.
This architecture uses Messenger as the messaging platform as discussed in Section 3.1. The architecture contains these key components: Messenger
Dialogflow
Firebase Function
Firestore Database
Scheduler v n o C d i
M l o o P d n E v n o C
Msg Msg Reply Ask for time
Time New conversation?
Request
Reply Ask for time Time reply
Webhook Function Webbhook Reply
Ask for time Time Reply
Save Save
Update Scheduling Database Initiate next day conversation
Save
{ Dialog ow is the core component of the implementation, where all the NLP,
intent classi cation, and entity extraction algorithms are taking place.
{ Firebase is where functions are implemented based on the requirement for
every single intent, for instance accessing a database to write or retrieve
data. Also, these functions are used for input veri cation, for instance, the
rating of an action should take an integer value between 1 - 5. The functions
are deployed in Firebase [
Most of chatbots cloud platforms, including dialog ow, are business-oriented and task-oriented, where the user is asking for speci c information such as what will tomorrow's weather be? This restricts the chatbot to only being responsive to users queries.
Our chatbot, Plybot needs to re-initiate a conversation session each day continuing from the previous session. Only Microsoft Bot Framework provides what is called proactive messages which can be sent based on a speci c event, for instance, a change in the price of a speci c commodity.
This feature is not implemented in dialog ow but the use of the Messenger API provides a solution where cron jobs are meant to send messages using the API at users speci ed times. These messages are constructed from context and information captured in the previous conversation session and therefore a continued conversation ow is possible.
Another important consideration for chatbot developers is testing their solutions. Although both dialog ow and Microsoft Bot Framework provide a cloud function deployments, relying on cloud deployment for testing a ful lment function is time-consuming as every deployment action can take a minute or more. In the developing stage, developers need the agility to test any change rapidly. The ability to perform tests using a local server with a public IP address is useful for development and testing.
Inconsistent quality in the cloud service platform API documentation slows development progress. For instance, Microsoft bot framework does not provide examples for integration with other platforms.
Cloud platforms have focused signi cant e ort on increasing security and
privacy. As a result, the privacy policies and system security parameters have
regularly changed since the platforms were introduced. For instance, the user id
was a standard eld in the dialog ow platform until it was deprecated on 30
June 2019, and after that date, the user needs to sign in using a Google
account through Google Assistance before the user id can be retrieved [
At the beginning of a conversation, dialog ow associate a session id that will remain active for a period of less than 20 minutes of inactivity. In applications such as Plybot, we should expect the users to discontinue the conversation and resume at their convenience. While this behaviour is not inherently supported by dialog ow, it was overcome by using a database to always keep track of the latest conversation stage to start from. 4
We successfully implemented Plybot, a chatbot that satis es the requirements for a clinical psychology-based research study as discussed in Section 2.1. It can act as a substitute for the practitioner in speci c tasks that are routinely in its nature, using a business-oriented cloud platform. A complete solution is implemented and deployed using dialog ow, rebase, restore and Messenger. Although platforms like dialog ow are not designed for domains such as healthcare, but it can still be used to achieve the requirements of this domain.
However, the cloud-based implementation means that our solution is at the mercy of the underlying platform for key features including proactive messaging and scheduling. Additionally, the rapid development cycle for the platforms make the implementation of a chatbot volatile and API changes may require updates to keep the Plybot solution working as backwards compatibility is not guaranteed.
From the QoE perspective, we present key observations that will be validated as a future work to be done along with the psychological study, except for the QoE for developer which is our subjective evaluation of the developing experience. These observations are as follows: { QoE for User: the use of Messenger allows easy access but limits the richness of UX widgets. Such implementation requires the trade-o between ease of no app to install against a better and more feature-rich UX, e.g. the use of dropdown menus for faster data input. The ability to interact with a chatbot from your phone rather than having to attend a practitioner interview is also a signi cant experience improvement. { QoE for Psychologist: it can be concluded that a chatbot implementation provides bene ts in terms of structured and consistent data gathering with fewer resources and interaction requirements. Ultimately, the e cacy of the chatbot as an alternative to traditional methods will not be known until the user study is completed. { QoE for developer: The initial bene ts of choosing a cloud-based solution (e.g. security, Messenger integration, NLP and AI, deployment) need to be weighed carefully against the platform restrictions (e.g. proactive messages) and maturity (changing APIs and documentation).
In practice, systems convert voice to text before it is processed. However, the e ect of using voice instead of text chatting can be addressed in further studies.
Future work will investigate enhancing the users' experience by adding elements that make the interaction more natural such as the typing indicator for a couple of seconds before the reply is sent. These types of enhancements may improve the sense of empathy. Dialog ow allows for only 5 seconds for a response to be received from a webhook function, and this limited time is not su cient to show the typing indicator before a reply is sent.
This publication has emanated from research supported in part by a research grant from Science Foundation Ireland (SFI) and is co-funded under the European Regional Development Fund under Grant Number 13/RC/2289 and Grant Number SFI/12/RC/2077. Thanks to Dr Louise McHugh and Alison Stapleton from UCD School of Psychology for input into the chatbot requirements.