We present an iterative and incremental approach to designing dialog management for a health coach chatbot based on our in-progress research. The requirements are derived from the coaching needs of young people living with HIV. We identify a hybrid dialog management approach to address diferent coaching needs as well as dialog acts to enable smooth conversations. In addition, relevant technical components were identified to be integrated into the dialogs to improve user experience.
Achieving a seamless, natural, and human-like conversation has been the aim of most conversational frameworks and platforms. A task that is intuitive for humans is not straightforward to implement in machines. To implement the various nuances of human conversations in dialog systems or conversational agents, most approaches have started with technology to identify an appropriate algorithm or technique that can address particular aspects of human conversation. For example, grounding mutual understanding, handling fallback situations, conversational repair, remembering the context of the conversation, etc., all contribute to the quality of conversational agents. Natural Language Processing has made it possible to use powerful algorithms to recognize speech, encapsulate conversations within a domain, and understand users’ goals, sentiments, and emotions. Thanks to these technology-oriented developments, most state-of-the-art conversational frameworks and platforms include in-built functionalities to handle some basic as well as advanced characteristics of human conversations.
A dialog manager is the central component of dialog systems or conversational agents
responsible for managing the context and the flow of conversation. Dialog Management
(DM) strategies have been broadly categorized as handcrafted/rule-based,
probabilistic/corpusbased/data-driven/end-to-end or a hybrid combination of both approaches [
Our scenario involves developing a health coach chatbot for young people living with HIV in urban and semi-urban areas in Nigeria. Twenty-three persons who are representative of this target group have contributed to the design of the chatbot and, due to their relevant role they are hereafter referred to as “champions”. The main purpose of the Health Coach chatbot is to guide and support the target group in various aspects of life and challenges related to living with HIV. The group seeks targeted and precise information as well as a non-judgmental partner who can empathize, motivate and engage with them while taking into account ethical and privacy considerations. It is not easy to classify a health coach chatbot as task-oriented or non-task oriented, as the task it performs or its goal is not explicit, nor is the conversation meant for entertainment. Given the dynamics of coaching, it is not straightforward to decide when to use rule-based DM, probabilistic DM, or a combination of both DM strategies. Moreover, we don’t know when to embed technology components like sentiment/emotion detection in the DM for a better user experience. Hence, we let ourselves be guided by the coaching needs of the champions to identify and embed appropriate technology in the DM of a health coach chatbot.
We, thus, identified the coaching needs of the champions and used them to design our conversations. We took a top-down approach to determine the technical requirements of the DM, which are derived from our conversational design. Our main contributions are 1. a combination of appropriate DM strategies to form a hybrid approach that addresses diferent coaching needs; 2. a smooth transition from one DM strategy to another to address multiple coaching needs in the same dialog; and 3. embedding appropriate technical components within the DM strategies to enhance user experience.
Our paper is structured as follows - in section 2, we discuss state-of-the-art literature related to DM; section 3 discusses our methodological approach; section 4 outlines the rationale of applying coaching in healthcare, specifically to support HIV. Section 5 discusses our implementation approach and evaluation. We discuss ethical considerations in section 6 and our future work in section 7.
An important functionality of the dialog manager is to manage the context by remembering the history of conversations and user-specific information. Most commercial conversational tools handle this feature but may use diferent terminology and/or technique in the background. Dialogflow [ 5] and IBM Watson remember the conversation state and use variables called contexts to control the conversation flow. In Rasa Core [ 6], the state and history of conversations are maintained in a tracker object. Voiceflow, the tool we use for DM, uses the concepts of flow and stack to manage the accessibility of intents and blocks, thus providing a smooth transition between hierarchies of nested dialog sequences [7].
Besides managing the dialog context, the dialog manager decides the next action in the
conversation. This decision is usually made by defining or learning dialog policy [
Considering DM approaches in general, both rule-based and data-driven approaches have pros and cons. A rule-based DM can be rigid and dificult to scale while data-driven approaches demand a huge volume of training data to be efective. Hybrid approaches have several advantages, allowing developers a degree of control while still requiring limited training data [3]. As examples, Quan et al. [9] have integrated a domain-specific pre-trained model with a rule-based DM, and Pichl et al. [10] have combined a tree-like rule-based DM with a generative language model to respond to out-of-domain queries.
In non-task oriented systems, the next action is an appropriate response to the user’s utterance, but in task-oriented systems, the response may be combined with concrete actions. There are several studies addressing the DM strategies in task-oriented systems. Common strategies are initiative, confirmation, and repair, especially in speech-based systems where problems arise in understanding or interpreting users’ speech inputs [8]. Whittaker et al. [11] have identified DM strategies for information presentation in the restaurant domain, like summarizing details, providing comparisons, and ranking recommendations. Williams et al. [12] have evaluated DM strategies for call routing, e.g., helping the users by asking open-ended questions, presenting a menu option, or directing them to a task by asking several questions. Since non-task oriented systems do not have a specific goal, conversations tend to be more open-ended and hence it is dificult to identify DM strategies. Yu et al. [ 13] have identified ten general DM strategies, e.g., switching topics, handling out-of-vocabulary words, ending a topic with an open question, etc., that can be applied to non-task oriented systems to avoid system breakdown. Often, a task-oriented conversational system also includes non-task oriented dialogs to improve user experience. A DM strategy for identifying the user’s goal for such dialog combinations, i.e., whether the user’s intention is to perform a task or chitchat, has been proposed by Nakano et al. [14]. Several recent works have identified dialog strategies through neural approaches [15, 16, 17, 18].
We adopted an iterative, user-centric approach to design our DM. We started by conducting a workshop to understand the coaching needs of the champions. Twenty-three champions in the age range of 18-25 were recruited and invited to share what they expect from a health coach chatbot. The requirements of the champions were mapped to the coaching needs and developed into conversation modules by public health experts, as described in section 5.1.
For DM, we use the open-source Dialog Manager API from Voiceflow 1. Voiceflow also provides a visual canvas for modeling rule-based as well as frame-based dialogs. For Natural 1https://www.voiceflow.com/api/dialog-manager Language Understanding, we use the NLU component from Rasa Open Source2, which allows for a flexible configuration of the NLU pipeline. The Health Coach chatbot will be accessed via WhatsApp3, which is one of the popular communication applications in Nigeria.
Figure 1 shows our in-progress iterative and incremental approach to DM. In the first iteration, all dialogs were designed in a rule-based manner, following a strict flow of conversation. In the next iteration, a frame-based DM was implemented by identifying certain slots within the conversations which can be filled from user utterances. This phase involved the identification of intents and entities that could be detected using the NLU component, which added a certain level of flexibility to the conversation flow. Further technical components like dictionary/lookup, pattern recognition for date/time entities, sentiment, and emotion detection were identified based on the coaching needs and integrated into the dialogs, thus enhancing the user experience.
The conversation modules underwent expert testing and subsequently, testing by the champions. Until the present stage, qualitative and quantitative feedback on user experience was collected, mapped to the technology, and implemented back in the conversation modules. Next iterations will focus on identifying and integrating relevant pre-trained language models, for example, models by Spacy for improved entity recognition4, model by ChatGPT for response generation5, models for emotion detection [19] etc. 2https://github.com/RasaHQ/rasa 3https://www.whatsapp.com/ 4https://spacy.io/models 5https://chat.openai.com/chat
Living with HIV involves several knowledge-related, attitudinal, and behavior challenges. These include, for example, adherence to treatment and medical supervision, dealing with stigma, and associated low levels of psychological well-being. HIV, like other chronic health conditions [20], causes a severe burden on healthcare systems in terms of efort and cost. Long-term support goes beyond meeting the medical requirements of a person and coaching is being increasingly adopted as a method to complement medical support. Coaching is a user-centric approach to bringing and supporting a sustainable lifestyle and behavior change by taking several factors into consideration [21]. The coaching process in healthcare includes activities like educating patients, improving their self-responsibility, stimulating active learning, etc. In the domain of HIV coaching, the main goal is to understand the challenges and barriers faced by patients and ensure that they adhere to the long-term treatment by regularly taking medications and showing up for appointments [22, 23, 24, 25, 26]. In most studies, coaches were trained on behavior change, communication, empathy, emotional intelligence, and motivation skills to achieve a long-term, reliable, and trusting relationship with the patients [21]. Besides these findings from the literature, the champions of this research expressed specific expectations, e.g., information on particular topics, encouragement, and a means to freely communicate without the fear of being judged. We combined the findings from the literature, the requirements expressed by the champions, and the feedback of the public health experts to derive the four categories of coaching needs addressed in our research.
In this section, we describe how the identified coaching needs influenced our conversation design and DM.
We grouped the coaching needs of the champions into four categories - Information and Knowledge, Communication and Engagement, Empathy and Motivation, and General Assistance. It is possible for a conversation to address more than one coaching need.
• The coaching need, Information and Knowledge (IK) is relevant because users have restricted options to obtain information due to the stigmatization of HIV. The need can be achieved through two types of conversations - 1. a detailed dialog on a particular topic and 2. short responses to a precise question, e.g., Frequently Asked Questions (FAQ). The ifrst category consists mainly of rule-based dialogues. Pre-defined button options help to focus on the main goal of the conversation, i.e., providing information on a particular topic. These conversations are generally informative but their scope for interaction is limited. Accordingly, embedded frame-based dialogs allow more personalized user interaction, and trigger reflective processes, e.g., by asking users to share their experience about a certain topic. To avoid monotony in responses, we used the “AI Assist” feature in Voiceflow that generates multiple variants using large language models (LLMs) in the background 6.
In the second category, users can ask for precise information without having to go through an entire topic. This functionality uses ML-based NLU and is implemented by defining FAQ intents and a lookup/dictionary feature that includes important keywords and domain-specific terminology. • Communication and Engagement (CE) need involves regular communication and dialog components that lead to improved engagement. In our dialogs, we designed game-like interactive elements such as calculating personal scores based on participants’ answers (e.g., a personal stress score that expresses the level of stress that a participant has to deal with), quizzes to verify the knowledge on certain topics, and interactive stories. In the initial iterations, these elements are integrated into specific conversation modules on specific topics, however, in the later iterations, users can initiate these elements independent of the topic. • In the coaching need, Empathy and Motivation (EM), the emphasis is on understanding the users’ emotional state and responding accordingly. This coaching need addresses several challenges of living with HIV, where understanding and empathy can bring about a positive change in people, like better adherence to medication, increased regularity in keeping appointments, and overall, aspirations to lead a healthy life.
In our dialogs, the understanding of users’ emotions has been achieved by integrating pre-trained machine learning models for understanding sentiments [27] and emotions [19]. The sentiment and emotion scores are combined with the intent confidence to present an appropriate response to the users. The initial empathetic responses were designed as a part of conversation design and variants generated using the “AI-Assist” feature of Voiceflow. This ensured that the variants have a similar tone and level of empathy as the response from which they were generated. • The final coaching need, General Assistance (GA), includes features like scheduling appointments, medication reminders, retrieving contact information on closest clinics, and emergency information. The scheduling feature is implemented as short dialogs combined with pattern recognition for date/time understanding and a custom scheduler to deliver the reminder. The contact information is implemented as an FAQ intent.
From the coaching-based DM approach described in the previous section, we defined several dialog acts, a selective list is shown in Table 1. Dialog Acts help in organizing the diferent actions that can be initiated either by the bot or the user or both and ensure a smooth transition between diferent topics and features provided by the chatbot to its users. Table 1 also shows a mapping between the dialog acts, the corresponding coaching needs addressed by the dialog act, and the technical components (if any) integrated into that action. In our iterative approach, some dialog acts have already been implemented and tested with the champions while some are planned for future iterations.
The greeting_<sentiment> dialog act is invoked when a conversation with the chatbot is started by a user and is a bot-initiated action by default. The act starts with the bot’s prompt "How are you doing?". The user’s response to this open question leads to an interpretation of the sentiment in it and is followed by an appropriate response from the chatbot. In a similar fashion, the respond_emotion dialog act together with sentiment detection is embedded within the conversations to respond based on the detected emotion.
Show Menu dialog act is invoked after the greetings and is also a bot-initiated action by default. Users can also go back to the menu from conversations by typing in the associated command. Since the chatbot will be accessed via WhatsApp, a main consideration is to not overcrowd the menu options on the screen. Thus, a hierarchical approach to organizing the available actions has been adopted. Additionally, restrictions by WhatsApp7 on the number of characters and buttons also have an influence on the design of the menu.
The diferent conversation modules developed on various topics can be triggered by the users through the show_<topicname> dialog acts. The FAQ acts are predicted when the user types in a query and are trained as dedicated FAQ intents. The emergency_contact is a type of FAQ that can either be queried by the user or suggested by the bot depending on the user’s responses to certain situations like side efects of medication.
The dialog acts related to medication and appointment reminders are a combination of botinitiated and user-initiated actions. These involve technical components like pattern recognition to understand the date/time of the reminder and a custom scheduler that will trigger the 7https://developers.facebook.com/docs/whatsapp/guides/interactive-messages/ reminders on the defined date and time.
Interactive actions like calculated scores (e.g., an indicator of stress or distinction between myth and reality), and quizzes are embedded within various conversation modules but can also be independently triggered by the users.
The escalate_human dialog act will be invoked by the bot when it captures signs of emergencies like depression or suicidal tendencies from the user responses. A custom-trained component will be used to identify such extreme signs and the user will be recommended to schedule an appointment with a medical/health counselor as early as possible.
The chat dialog act will be invoked by the user when they wish to have a free conversation with the chatbot, i.e., without choosing the pre-designed conversation modules. This functionality will use end-to-end DM, as discussed in section 7.
The dialog acts together with the session information will allow the users to move smoothly between the conversations. Figure 2 shows examples of dialog acts calculate_score, greetings and an interactive story.
The evaluation for this study will also be carried out iteratively using methods like expert evaluation, topic-based evaluation, and other relevant approaches as described by Deriu et al. [28].
The prototype phase is evaluated through expert feedback and by collecting quantitative and qualitative feedback from the champions, leading to a continuous improvement of the chatbot prototype. Current metrics include measuring new knowledge and engagement/entertainment delivered through the content. Qualitative feedback collects the strengths and improvements of the conversational experience. 8GIF source: https://giphy.com/gifs/titanic-Uj3SeuVfg2oCs
The Health Coach chatbot will be rolled out to larger groups in a phased manner and a mixedmethod evaluation will be carried out. The evaluation will include a randomized controlled trial to measure the use, usability, and perception of the Health Coach chatbot, besides other metrics related to adherence, psychological well-being, and acquired knowledge.
The ethical considerations for studies related to HIV are very high and participant privacy is of utmost importance. Fisher et al. summarize guidelines for ethical consideration related to participant recruitment, data maintenance, and consent [29]. Our study includes relevant steps to ensure the privacy of the participant and their data. The participants are educated on mobile use, privacy, and security of their devices. The data is encrypted, anonymized, and stored in secure data centers located in Switzerland. Pre-trained models are hosted on own infrastructure to ensure that conversations with the chatbot are dealt with securely. Additional precautions include avoiding terms like ‘HIV’ and ‘medication’/‘ARV’ and using placeholders like ‘H’ and ‘sweets’, respectively, as can be seen in one of the examples in Figure 2.
The recent introduction of generative models like GPT-39 and ChatGPT provides an opportunity to tackle dificult tasks like generating training conversational data for HIV coaching and response variants but poses another ethical challenge as the generated text may be factually incorrect or sometimes non-sensical. It is important that the generated text is verified by human experts before integrating it in an application. For this reason, we do not consider answering the participant queries in real-time through the said generative models. However, we do see the benefits of using these models in an ofline manner after a thorough review and verification of the generated content.
As a continuation of our iterative approach, we will integrate the following components in our DM: • End-to-end DM: One of the requirements expressed by the champions is to be able to chat freely with the chatbot. This requirement can be linked to coaching need Empathy and Motivation as the champions view HIV coach chatbot as a non-judgmental conversational partner. This requires a very powerful understanding on the chatbot’s part and can be achieved through end-to-end DM. As described earlier, end-to-end DM is a data-driven approach where user intents are not explicitly defined or recognized. Using probabilistic methods, an appropriate response to user utterance is identified from training data. However, the implementation of end-to-end DM has several challenges. It requires a vast amount of training data in the form of past conversations. In the context of HIV coaching, it is not easy to access past conversations and one must resort to generating training data. For an efective conversation, the chatbot should also understand small talk besides domain-specific utterances to enable bonding. An approach in this research will be to use the conversation modules designed for various HIV-related topics as training data. In addition, the training data can be augmented with conversations generated by tools like ChatGPT and verified by human experts. Technically, this will be achieved using the End-to-End Training feature in Rasa10. • Human-in-the-loop escalation: From the feedback of our public health experts, several situations for human escalation have been identified. E.g., when cases of depression, suicidal tendencies, or other extreme symptoms are detected from the responses of the champions. Currently, the chatbot advises the champions to contact a counselor as soon as possible, which may result in hesitance, postponement, or even negligence in visiting the counselor. This action could be supported by going a step further and helping the champions to schedule an appointment, e.g. by sending a message on WhatsApp or by sending an email to a counselor and ensuring follow-up with the champions in question.
In this paper, we presented our iterative approach to dialog management for a HIV coach chatbot, being developed in collaboration with public health experts in Nigeria. We have taken a top-down approach by identifying the long-term coaching needs of young people living with HIV by conducting workshops with the participants of this study. Starting with a rule-based conversational design, we have identified the next steps to improve the interaction as well as the conversational experience of the users of the Health Coach chatbot. The choice of technical components like pre-trained machine learning models, training data as well as generative models is made depending on the user requirements. We have also outlined our approach to continue the development of the chatbot for the future phase of the ongoing project. The Health Coach chatbot will be continuously evaluated in a phased manner as a randomized controlled trial, taking ethics and privacy into consideration at every step of the project.
This work has been funded by the Swiss National Science Foundation (SNSF) under grant IZSTZ0_202602 within the Swiss Programme for International Research by Scientific Investigation Teams (SPIRIT).
We thank Voiceflow, Inc. for providing us access to their conversation design platform. 10https://rasa.com/docs/rasa/next/training-data-format/#end-to-end-training [2] D. Jurafsky, J. H. Martin, Chatbots and Dialogue Systems, in: D. Jurafsky, J. H. Martin (Eds.), Speech and Language Processing, draft 3rd ed., Stanford and Colorado at Boulder, 2023. URL: https://web.stanford.edu/~jurafsky/slp3/. [3] H. Brabra, M. Báez, B. Benatallah, W. Gaaloul, S. Bouguelia, S. Zamanirad, Dialogue management in conversational systems: a review of approaches, challenges, and opportunities, IEEE Transactions on Cognitive and Developmental Systems (2021). [4] S. Hussain, O. Ameri Sianaki, N. Ababneh, A survey on conversational agents/chatbots classification and design techniques, in: Workshops of the International Conference on Advanced Information Networking and Applications, Springer, 2019, pp. 946–956. [5] Google.com, Dialogflow, 2023. URL: https://cloud.google.com/dialogflow, (Accessed on 13/01/2023). [6] T. Bocklisch, J. Faulkner, N. Pawlowski, A. Nichol, Rasa: Open source language understanding and dialogue management, arXiv preprint arXiv:1712.05181 (2017). [7] Voiceflow.com, State, 2023. URL: https://developer.voiceflow.com/docs/state, (Accessed on 01/11/2023). [8] J. Schatzmann, K. Weilhammer, M. Stuttle, S. Young, A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies, The knowledge engineering review 21 (2006) 97–126. [9] J. Quan, M. Yang, Q. Gan, D. Xiong, Y. Liu, Y. Dong, F. Ouyang, J. Tian, R. Deng, Y. Li, et al., Integrating pre-trained model into rule-based dialogue management, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2021, pp. 16097–16099. [10] J. Pichl, P. Marek, J. Konrád, P. Lorenc, O. Kobza, T. Zajíček, J. Šedivy`, Flowstorm: Opensource platform with hybrid dialogue architecture, arXiv preprint arXiv:2212.09377 (2022). [11] S. Whittaker, M. Walker, Evaluating dialogue strategies in multimodal dialogue systems, in: Spoken Multimodal Human-Computer Dialogue in Mobile Environments, Springer, 2005, pp. 247–268. [12] J. D. Williams, S. M. Witt, A comparison of dialog strategies for call routing, International
Journal of Speech Technology 7 (2004) 9–24. [13] Z. Yu, Z. Xu, A. W. Black, A. Rudnicky, Strategy and policy learning for non-task-oriented conversational systems, in: Proceedings of the 17th annual meeting of the special interest group on discourse and dialogue, 2016, pp. 404–412. [14] M. Nakano, A. Hoshino, J. Takeuchi, Y. Hasegawa, T. Torii, K. Nakadai, K. Kato, H. Tsujino, A robot that can engage in both task-oriented and non-task-oriented dialogues, in: 2006 6th IEEE-RAS International Conference on Humanoid Robots, IEEE, 2006, pp. 404–411. [15] M. Huang, X. Zhu, J. Gao, Challenges in building intelligent open-domain dialog systems,
ACM Transactions on Information Systems (TOIS) 38 (2020) 1–32. [16] T. Zhao, K. Xie, M. Eskenazi, Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models, arXiv preprint arXiv:1902.08858 (2019). [17] J. Ni, T. Young, V. Pandelea, F. Xue, E. Cambria, Recent advances in deep learning based dialogue systems: A systematic survey, Artificial intelligence review (2022) 1–101. [18] L. Galland, C. Pelachaud, F. Pecune, Adapting conversational strategies in informationgiving human-agent interaction, Frontiers in Artificial Intelligence 5 (2022) 1029340. [19] J. Hartmann, Emotion english distilroberta-base, https://huggingface.co/j-hartmann/ emotion-english-distilroberta-base/, 2022. [20] T. W. Mahungu, A. J. Rodger, M. A. Johnson, Hiv as a chronic disease, Clinical Medicine 9 (2009) 125. [21] R. Q. Wolever, M. A. Moore, M. Jordan, Coaching in healthcare, The Sage handbook of coaching (2017) 521–543. [22] D. Swendeman, E. M. Arnold, D. Harris, J. Fournier, W. S. Comulada, C. Reback, M. Koussa, M. Ocasio, S.-J. Lee, L. Kozina, et al., Text-messaging, online peer support group, and coaching strategies to optimize the hiv prevention continuum for youth: protocol for a randomized controlled trial, JMIR research protocols 8 (2019) e11165. [23] K. R. Amico, J. C. Lindsey, M. Hudgens, R. Dallas, K. J. Horvath, A. Dunlap, R. Goolsby, M. M. Johnson, B. Heckman, J. Crawford, et al., Randomized controlled trial of a remote coaching mhealth adherence intervention in youth living with hiv, AIDS and Behavior (2022) 1–17. [24] N. A. Haug, J. L. Sorensen, V. A. Gruber, N. Lollo, G. Roth, Haart adherence strategies for methadone clients who are hiv-positive: a treatment manual for implementing contingency management and medication coaching, Behavior modification 30 (2006) 752–781. [25] S. E. Ramsey, E. G. Ames, J. Uber, S. Habib, S. Clark, D. Waldrop, A preliminary test of an mhealth facilitated health coaching intervention to improve medication adherence among persons living with hiv, AIDS and Behavior 25 (2021) 3782–3797. [26] M. Hlongwa, M. Cornell, S. Malone, P. Pitsillides, K. Little, N. Hasen, Uptake and short-term retention in hiv treatment among men in south africa: the coach mpilo pilot project, Global Health: Science and Practice 10 (2022). [27] C. Hutto, E. Gilbert, Vader: A parsimonious rule-based model for sentiment analysis of social media text, in: Proceedings of the international AAAI conference on web and social media, volume 8, 2014, pp. 216–225. [28] J. Deriu, A. Rodrigo, A. Otegi, G. Echegoyen, S. Rosset, E. Agirre, M. Cieliebak, Survey on evaluation methods for dialogue systems, Artificial Intelligence Review 54 (2021) 755–810. [29] C. B. Fisher, E. Bragard, R. Bloom, Ethical considerations in hiv ehealth intervention research: Implications for informational risk in recruitment, data maintenance, and consent procedures, Current HIV/AIDS Reports 17 (2020) 180–189.