“Do Users Need Human-like Conversational Agents?” – Exploring Conversational System Design Using Framework of Human Needs Souvick Ghosh1 , Satanu Ghosh2 1 San José State University, One Washington Square, San José, 95192-0029, CA, United States 2 University of Oklahoma, School of Library and Information Studies, 401 West Brooks, Norman, 73019-6032, OK, United States Abstract The fascinating story of human evolution can be attributed to our ability to speak, write, and communicate complex thoughts. When researchers envision a perfect, artificially intelligent conversational system, they want the system to be human-like. In other words, the system should converse with the same intellect and cognition as humans. Now, the question which we need to ask is if we need a human-like conversational system? Before we engage in the complex endeavor of implementing human- like characteristics, we should debate if the pursuit of such a system is logical and ethical. We analyze some of the system-level characteristics and discuss their merits and potential of harm. We review some of the latest work on conversational systems to understand how design features are evolving for Conversational Agents. Additionally, we look into the framework of human needs to assess how the system should assign relative importance to user requests, and prioritize user tasks. We draw on the peer work in human-computer interaction, sentiment analysis, and human psychology to provide insights into how future conversational agents should be designed for better user satisfaction. Keywords Conversational Agents, Smart Agents, Need Based Design, Maslow’s Hierarchy 1. Introduction USD in 2020 to 13.9 billion USD by 20231 . Conversational agents are now used by several commercial sectors for As humans, we are fascinated with anything that can rendering support related to healthcare [1], education [2], talk, walk, or behave as humans do. While it is true that elderly care [3], customer service [4], and information any intelligent being should be able to communicate, the retrieval [5]. Some of these systems are voice-based only forms of communication may vary. For a system to in- and also known as intelligent personal assistants (IPAs). teract with humans efficiently, it should speak and write A few popular systems present in the market now are in a manner which is easily understood by the human Amazon’s Alexa2 , Google’s Assistant3 , Microsoft’s Cor- users. In the late 18𝑡ℎ century Erasmus Darwin invented tana4 , and Apple’s Siri5 . a machine that could produce single phonemes and this With the increase of preference for humanoid systems, was probably the first successful attempt of constructing researchers and developers have been increasingly de- a machine that could produce human sounds. Around the voted to designing conversational systems which are 1960s, researchers started exploring the idea of a talking more anthropomorphic, or human-like. Human voices – computer. With time, our understanding of science and with options of selecting from multiple speakers, genders, technology developed, and we developed computational and dialects – have replaced robotic voices. To increase systems that can talk and understand natural language. the novelty factor and attractiveness of these systems, Intelligent Personal Assistants (IPAs) have flooded the celebrity voices are also being used. Research attempts market commercially and have become part of our every- are also being made towards more user-friendly and ac- day lives. We use them on the phone, on smart speakers, cessible user interface, better system-level cognition and and in our cars. It is predicted that the market value response, organic development of natural language di- of AI conversational systems will rise from 4.8 billion alogues, and effective ways of presenting the retrieved information. Overall, the above mentioned research di- rections should help in developing conversational sys- DESIRES 2021 – 2nd International Conference on Design of tems which recognizes user sentiment and responds with Experimental Search Information REtrieval Systems, September empathy. 15–18, 2021, Padua, Italy " souvick.ghosh@sjsu.edu (S. Ghosh); satanu.ghosh-1@ou.edu 1 https://www.marketsandmarkets.com/Market-Reports/ (S. Ghosh) conversational-ai-market-49043506.html ~ https://souvickghosh.com/ (S. Ghosh) 2 https://developer.amazon.com/en-US/alexa  0000-0003-1610-9038 (S. Ghosh) 3 https://assistant.google.com/ © 2021 Copyright for this paper by its authors. Use permitted under Creative 4 Commons License Attribution 4.0 International (CC BY 4.0). https://www.microsoft.com/en-us/cortana CEUR Workshop Proceedings (CEUR-WS.org) 5 CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 https://www.apple.com/siri/ Interaction between users and conversational systems the user, who can explain his information need with long are task- or goal-oriented, and often with a definite set descriptions and more context. of objectives. It could include but not be limited to con- The medium of interaction between the user and the trolling smart home devices – switching on the lights, system influences the design and application of the sys- setting an alarm, turning up the temperature. The users tem. A text-based conversational system is referred to also prefer to ‘talk’ to the system, treating it as a human as a chatbot while a voice-based system is called a per- conversational partner. This type of user behavior could sonal assistant (or intelligent personal assistant). Use of be encouraged by system novelty or user boredom. Eval- voice enables spoken systems to be used in hands-free uation of such task-based systems are often governed by and eyes-free situation, which is common while driving, the success or failure of the user tasks or fulfillment of the cooking, or working out. All of these situations involve a user objectives, therefore, the system needs to prioritize primary task which is the focus of the user attention. The tasks of higher importance over others. conversation is secondary and is employed to achieve In this paper, we survey some of the latest papers ex- simple tasks or question answering. A chatbot, however, ploring humanoid features for conversational systems. allows collaboration among multiple users, and presen- The review helps us assess the potential merits and harms tation of lists, images, and videos. Since text allows the of implementing the researched characteristics. Next, we user to scan, the system response can be longer and more use Maslow’s hierarchy of human needs [6] to suggest detailed. A multimodal system like an embodied con- how the system should prioritize between multiple tasks versational agents (ECA) [7, 8, 9] has a virtual face or and assign importance to different user needs. Lastly, we body (artificially generated) – in addition to text or voice use some use case scenarios to demonstrate how the ex- – and can therefore, communicate using facial expres- isting system may adversely affect the users’ interaction sions, gestures, body language, and non-verbal cues. The experience. ability to display sentiment makes ECAs applicable to The rest of the paper is organized as follows: In Section mental health domains where the system should be able 2, we review some of the system-level human-like char- to empathize and display emotions. The last decade have acteristics which were implemented in conversational witnessed massive popularity of mobile devices, which agents. In Section 3, we look into Maslow’s Need Hier- has provided a perfect platform for voice-based system. archy and its application in the design of conversational Conversational systems have already found application systems capable of prioritizing user tasks. Lastly, we in searching [10, 11], flight booking services [12], and present use case scenarios to highlight some of the issues vacation planning [13, 14]. To address loneliness in pa- with existing systems and how the need hierarchy could tients, conversational systems have also been deployed be utilized to mitigate them. In Section 4, we conclude as conversation partners [1]. However, modern day con- the work and propose future directions. versational systems are still in a developing stage and more research is required before mimicking the complex nature of human-human conversations. In the following 2. Exploration of System-level subsections, we look into some of the design aspects (or Characteristics characteristics) introduced by researchers for different types of conversational systems. The popularity of conversational systems – where the conversation could be voice-based or text-based – can 2.1. Personality be largely attributed to their ability to understand and generate natural language dialogues. A successful design Personality can be defined as a set of characteristics that is one which seamlessly integrates with the environment determine how a person behaves or reacts to their en- and the system is almost invisible to the user. A major vironment. For characterizing human personalities, re- application of conversational systems is in information searchers often use the Big Five Model or the OCEAN retrieval, where the user can approach the system with model. The OCEAN [15, 16] is an acronym composed his query, and the system responds with the useful infor- of five different personality types: Openness, Conscien- mation. However, the user-system interaction in search tiousness, Extraversion, Agreeableness, and Neuroticism. systems is fraught with problems. First, the user has to Multiple studies [17, 18, 19] have used this model to ex- represent his information need using a set of keywords plain how personalities should be designed for conversa- (queries). Moreover, towards the beginning of the search tional agents. session, the user is not cognizant of the exact nature of Braun et al. (2019) [20] suggested that the user wants his information problem. The problem of cold start has the agent to reflect the users’ personality. Other studies been researched in the information community and the found that extraversion is the most common user per- cognitive load placed on the user is far from ideal. Use sonality, which was also found in conversational agents. of natural language should reduce the cognitive load of Neff et al. (2011) [21] reported that the users sensed neuroticism in the CA. However, many researchers feel 2.3. Voice that the OCEAN model is neither sufficient nor appro- While voice is not a mandatory element in conversa- priate to model CA personalities and have proposed an tional agents, it makes the conversation more natural. alternate model of personalities [22] – the Three-Factor Voice-based systems allow the users to multitask while Model [23] – which is quite different from the OCEAN. performing a primary task (driving or cooking). Spoken Another study [24] discusses how the language used by dialogues are the natural form of communication and conversational agents can influence the way users per- promote trust between the participants [42]. User com- ceive their personality. The locale and cultural practices fort and satisfaction increase significantly when the user also play an important role in how the users perceive can trust the agent. Since emotion is an important aspect the agent. Even slight variations of tone and acoustics of the human-human conversation, and facial, bodily, or can play an essential role in user perception. Kim et al. gestural expressions are not always available, the impor- (2019) [25] categorized personality traits into Common tance of voice is magnified in a user-agent interaction. Trait, Distinctive Trait, and Neutral Trait. The channels Nunamaker et al. (2011) [43] reported that users perceive of expression (of personality) were also divided into three certain gender to be more trustworthy, able, or likable. categories: Ways of Providing Service, Types of Service, Therefore, in a voice-based environment, the choice of and Language & Appearance. The more popular and voice (male or female), the pitch, and the loudness can ubiquitous conversational agents like Alexa [26] were affect the user’s perception of the agent. Also, by uti- designed to be smart, approachable, humble, enthusiastic, lizing the acoustic and prosodic properties of voice, the and helpful. In contrast, Siri was designed to be friendly agent can identify user emotions and express its feelings. and humble, but with an edge to her personality [27]. Danielescu and Christian (2018) [24] found that users want more control and would prefer to select the type of 2.2. Empathy voice of the agent. In any conversation, emotions indicate the level of en- gagement and satisfaction. Therefore, if we consider 2.4. Embodiment individual utterances, the emotions displayed should be In addition to voice, conversational agents can also have considered while generating the agent response. Simi- an artificially generated face or body. Research on em- larly, the emotion expressed in the agent response can in- bodied agents suggests that a body and voice can help fluence the user engaged in the discourse. Empathy is an users to socially accept an agent. The presence of ver- essential socio-emotional behavior for effective interper- bal and non-verbal cues in embodied agents allows for sonal communication. During a conversation, humans the expression of empathy [25] and emotions. Embodied often exhibit affective empathy [28] which can be de- agents – using multimodal channels – are perceived to fined as the human nature of automatic and unconscious be more socially present for the users [44, 45]. Some mimicking of the other participants to match or mirror studies [8, 9] claim that human-like intelligence can only their emotions. On the other hand, for cognitive empathy be exhibited by artificially intelligent systems through [29], we consider the perspective or mental state of the non-verbal cues, and that is only possible when using an conversational partner before reacting. A general ten- embodied conversational agent. Rheu et al. (2021) [46] dency of researchers is to create conversational agents suggested that embodiment can make the agent more which are sympathetic [30, 31], supportive [32, 33] or trustworthy. compassionate [34]. Few agents exhibit qualities like af- Despite the affordances offered by embodied conver- fective matching [35, 36] and mirroring mechanism [37]. sational agents, we must look at the potential of harm. Complex models like the EMMA framework [38] and Gender of the embodiment has also been a topic of dis- the CARE framework [39] have also been developed for cussion among researchers. While some researchers agents. However, if we look at the existing state-of-the- think that androgynous personas will contribute toward art assistants, they are not empathetic. Since empathy unbiased agents, others argue that humanizing agents consists of multiple layers [40], the implementation of will lead to better performance. User interactions with complex empathetic expressions become extremely chal- female embodied agents resulted in more sexual and lenging. Integrating human-like empathetic responses in swear words [47]. The effect of gender was also ob- a conversational system could be thought of as a three- served in other studies where the users perpetuated level process: perspective-taking, context generation, and gender stereotypes specific to agent personality and expression. Perspective-taking [41] means understand- roles [48, 49, 50, 51]. While embodied agents can be ing the views, beliefs, desires, and intentions of the user. designed with the option to stop any gender manifesta- tions [52], it can also alienate the user from the agent and lead to fewer interactions. 2.5. Ethics the dialogue style [67] and voice parameters could be adjusted to meet the user’s preferences. In any type of discourse, language is a primary compo- Personalization efforts can be grouped under two nent that reflects the political, sociological, and cultural broad categories. The agent has to either store infor- conditions of a particular time [53]. The choice and usage mation from every interactive session (implicit personal- of words are governed by the context, and the synchronic ization) [68] or ask the user a set of questions (explicit nature of language [54]. A word that is deemed accept- personalization) [69] at the beginning of every session. able at present may not be acceptable in the future. If The two approaches present a trade-off between conve- we look at the word ‘awful,’ it has a negative meaning nience and privacy. associated with it. However, it originated as a shortened One important aspect of personalization is the res- form for “full of awe,” which is a positive phrase (refer- olution of conversational implicatures in human-agent ring to something that inspires wonder). Therefore, a dialogues. Conversational implicature is an important lin- conversational agent cannot focus only on the linguistic guistic phenomenon that allows humans to imply mean- aspects and ignore the socio-cultural contexts. ings without clarifying them explicitly [70]. It helps to Ethics depends on four major factors: time, context, keep the conversation short and hedge negative emo- user perception, and user’s socio-cultural aspects. Ethics tions. Such implications are also common for users who in conversation involves knowing which words to use are depressed or suicidal [7]. Yule (2020) [71] shows how in a dialogue and which words to avoid. Modern-day complex implications are difficult to understand, even in conversational agents are deployed in the field of mental human-human conversations. Since existing agents lack health [55], where they talk to people and make them sufficient cognition to interpret implications in human feel better. Since people with mental health issues are dialogues, they must ask clarifying questions and resolve vulnerable, the conversation should be carefully struc- ambiguities. Such clarifications increase the number of tured to avoid hurting anyone’s sentiments. Kretzschmar turns and may lower user engagement. One possible et al. (2019) [56] discussed that such agents do not always solution is to save dialogues from previous interactions consider the potential of harm. with the user. Schlesinger et al. (2018) [57] found that some agents However, while personalization and implicature reso- use a collection of blacklisted words to detect undesir- lution will lead to better user experience, it comes at the able speech and, therefore, deflect questions related to cost of reduced privacy. Interactions with conversational race. However, users often perceive this deflection as agents – using natural free-form language – can lead an endorsement of racial hate or nonchalance toward to the disclosure of personal and sensitive information racial issues. When the user initiates an open-domain related to health, security, or finance. Saffarizadeh et al. conversation on sexual harassment, some agents [58] (2017) [72] believes that more users prefer privacy over even responded with counter-aggression or flirtatious be- personalized response. An acceptable solution would havior. While such behavior can be attributed to training be to allow the users to decide how much personal in- data, the developers cannot ignore the lack of ethics in ex- formation they want to share and their desired privacy isting conversational agents. Whittaker et al. (2018) [59] levels. The agent should disclose all the signals it has and O’neil (2016) [60] argue that the developers often collected from the user, both implicitly and explicitly. ignore ethical considerations in favor of technical aspects. Also, the collected information should be encrypted to Conversational agents must be evaluated continuously or prevent unauthorized access. Should the user exercise periodically to confirm that the systems are not behaving the right to be forgotten, the agent must clear all the unethically. To use such agents in everyday life – where user’s stored data. While this would alleviate privacy they interact with humans – the potential harm must be concerns, it adversely affects the personalization efforts. mitigated. Any attempts by the agent to collect data without the user’s consent could be perceived as a threat [73] and 2.6. Personalization & Privacy reduce user satisfaction. After an extensive exploration of existing literature Increasing personalization allows the agent to behave on intelligent conversational agents, we identified one uniquely for every user, tailoring the agent’s decisions to major shortcoming in current state-of-the-art systems. the personality and preferences of the user. Personaliza- The inability of the agents to prioritize tasks is a major tion enables the agent to dynamically adapt to the user challenge for intelligent conversational agents. There- and make better recommendations, which increases user fore, in the following section, we propose a framework satisfaction [61, 62, 63]. Several studies have discussed that can be used to prioritize user tasks. how personalized agents can be more effective in health- care [63], libraries [61], business [64] and education [65]. The relevancy of results could be improved [66], and 3. Prioritization of User Tasks using Maslow’s Need Hierarchy Our review of existing conversational systems high- lighted that although several intelligent functionalities have been proposed and implemented in existing sys- tems, the system design is still not ideal for context me- diated behavior and task prioritization. Prioritizing tasks - by intelligent agents - is essential to guarantee a faster turnaround time with greater accuracy for tasks of higher priority which can ensure better user satisfaction. There exists a strong relationship between user satisfaction and the design of conversational systems. In order to widen the scope and application of these systems, it is essential to look into the human aspect of such systems in addition Figure 1: Maslow’s Hierarchy of Needs. to the computational side. For example, insights from hu- man psychology [74] – such as the framework of human needs – could significantly improve the operationaliza- tion and functioning of conversational systems. with the three new levels which were added later. The development and design of any system is borne • Physiological Needs: out of need to solve a problem, or to improve an existing The most primitive needs for any living organism solution. Therefore, any system design should concen- (including humans) are to survive and grow. The trate on the needs of the end user and the potential of requirements for sustaining life include food, wa- the system to satisfy those needs, either working on its ter, air, and sleep. Once survival is ensured, the own or in collaboration with the user. focus shifts to maintaining optimal conditions for Maslow [75] looked into the story of human evolution growth. and proposed a hierarchical framework to explain how • Safety Needs: different needs are prioritized by the human mind. The Safety requirements are the second most impor- human mind is motivated by the instincts to survive, both tant need for humans. This means safety for self as an individual and as a species. Therefore, it assigns and those who are closest to them. Safety is of- varying levels of importance to the things around us. The ten connected to a known order. Therefore, any level of satisfaction is higher if a higher order need is unpredictability or course of events which could satisfied. For example, any primitive organism aims to pose a threat to life, or living conditions of an secure the basic items which it needs to survive. This individual, is detrimental to the user experience could include food, water, air, and temperature optimal and satisfaction. for growth. This is no different for a human baby. Any potential threat to survival is met with the desire to fight • Community and Belonging: or flight. As an organism evolves – evolution of life For a majority of humans, their existence is not or a baby growing into an adult – the basic needs are merely as an individual but as a part of a larger supplemented by higher order needs. Such needs – which social group or community. Humans cherish the could be philanthropic, spiritual, or materialistic – are emotional connection and togetherness which not replacements for basic needs. The fundamental needs they feel with their family, friends, colleagues, are still important for survival but the higher order needs and romantic partners. Community and belong- go beyond the needs of the self. ing needs allow humans to avoid loneliness and The tier-based structure of Maslow’s Need Hierarchy leads to psychological well-being. The transition contains five levels and is shown in Figure 1. The different from individual survival to community needs oc- levels in the hierarchy suggests that before an individual curs only when there is no threat to the underly- pursues any top-level needs, he must ensure that the ing need levels. fundamental needs (those related to survival) are satisfied. • Esteem Needs: The two levels at the bottom represents psychological Esteem needs are related to the feeling of self- and safety needs which are essential for survival. This is worth and can be broadly categorized into two followed by two more levels of psychological needs (love categories: self-respect and respect from peers and esteem). Finally, we see the need self-actualization and community. Humans possess a strong desire at the top. We explain these five levels in detail, along to be accepted, appreciated, and validated by their social circles. They also value freedom and choice, and feeling confident and competent. A system and fulfilled from bottom upwards. Therefore, only when should respect the esteem needs of the users as the primary needs like hunger and thirst are satisfied, any action detrimental to esteem needs will likely does the individual look for “higher” needs. There is also lower the satisfaction levels of the user. an inherent relationship between human motivation and • Cognitive Needs: needs. Maslow suggested that for the fundamental or While cognitive needs were not part of the ini- basic needs (physiological and safety needs like air, water, tial need hierarchy, they were later added to the food, shelter), as the deficiency increases, the motivation original five levels. These needs are the dominant increases as well. Therefore, these needs as strongest mo- reason why humans strive to acquire more knowl- tivators for any human being. However, the fulfillment edge and challenge their intellect by partaking in of these needs results in decrease in motivation. A man cognitively complex tasks. who has sufficient bread to eat does not strive for more • Aesthetic Needs: bread. Instead, they look for higher-order needs (love, Aesthetic needs were appended to the original esteem and self-actualization). However, fulfillment of need hierarchy. These needs are symbolic of higher needs does not lower motivation. Instead, motiva- the human fascination to create and appreciate tion keeps increasing as these needs are being met. This beauty, artistic marvels. explains why humans strive for more fame, money, and • Self-Actualization: achievements, although they have enough. In the original five-tiered need hierarchy, self- While the framework of human needs proposed by actualization was at the top of the hierarchical Maslow is hierarchical, researchers have debated if the pyramid, which means that the fulfillment of different levels are mutually exclusive. Also, how often this need is prioritized after all the lower level are the different levels pursued simultaneously? When needs are met. Self-actualization is the urge to there is no food or water (‘bread’ as Maslow calls it), reach one’s true potential and fulfill one’s talents. and hunger is the motivating factor, a man will prioritize The self-actualization needs could be fulfilled by the need for food above others. But that does not stop gaining knowledge, receiving awards in one’s do- him from looking for a safe shelter, obtaining education, main of work, and fulfilling long- and short-term or looking for a better job. While the need levels may dreams. The achievements of goals by the indi- be clearly distinguishable from each other, the actions vidual are metrics to judge how well the needs may not be. Certain actions may fulfill the lower-level were met. needs in the short-term and higher-levels in the longer • Self-Transcendence: run. However, it can be agreed upon that social, cultural, Like cognitive and aesthetic needs, self- and economic aspects (which are specific to every indi- transcendence was added later to the needs vidual) governs how a human prioritizes the different hierarchy. These needs are mostly spiritual in needs. Deficiency in a lower-order need may act as a nature and connects the individual to a higher deterrent to pursue higher needs, but there are many purpose or entity. Spiritual needs, which may exceptions. Many high-achieving individuals, the basic or may not be connected to organized religion, needs are sacrificed to fulfill esteem and self-actualization transcend the materialistic pleasures and gives needs. For others, the needs are in a state of constant meaning to the life of individuals. change throughout the life of the individual. The basic In the words of Maslow: needs (in infants) are supplemented by safety and love needs (as they mature), and esteem and self-actualization It is quite true that man lives by bread needs (when adults). Certain life situations (financial alone — when there is no bread. But what hardship, health complications) may motivate some of happens to man’s desires when there is the needs more than others but for every individual, there plenty of bread and when his belly is is a unique balance between the different needs. The chronically filled? At once other (and needs framework, while being hierarchical, has varying “higher”) needs emerge and these, rather amounts of overlap depending on the individual. than physiological hungers, dominate the As we discussed in Section 2, many advanced features organism. And when these in turn are sat- have been implemented in current conversational agents. isfied, again new (and still “higher”) needs Surprisingly, none of these features help to understand emerge and so on. This is what we mean the context - from user utterance - and prioritize tasks by saying that the basic human needs are accordingly. For example, the acoustic properties of user organized into a hierarchy of relative pre- utterance changes with the user’s mood and situation. potency. (Maslow, 1943, p. 375) [6] Therefore, an intelligent agent should use such features When Maslow [6] proposed the hierarchical needs to determine the context of the task. Subsequently, the framework, he conceptualized the needs to be prioritized contextual information can be utilized to decide on task priority and the agent action. There is a break in. Agent: Sorry, I do not understand. User experience: Kevin realized that the agent is 4. Use Case Scenarios with failing to recognize his panicked voice. He found Voice-Based Conversational his phone to call law enforcement. • Situation 3: Tarek is lonely and struggling with Systems health issues. He decides to talk to the agent In Section 2, we discussed the various human-like charac- about his health condition. teristics which the users desire of conversational agents. Need: Love and Belonging (Emotional Support) While each of those functionalities will require careful Kevin: Hey , I am having trouble development (so as to avoid any potential for harm), the with responses and urgency of the agent should consider the Agent: This is what I found. relative importance of human needs. Our exploration User experience: Tarek is upset because the agent of Maslow’s framework [6] suggested the user have lim- not only failed to maintain conversation but inter- ited patience when their basic needs are threatened. The rupted him and provided irrelevant information. range of use cases could vary from an user looking for There is strong probability that he will not use shelter homes, community food kitchens to another look- the agent in the future. ing for a nearby restaurant. The physiological needs • Situation 4: Tina is not a native speaker of English have varying degrees of importance and the users need but prides herself in being fluent in English. She the agent to be empathetic to their problem. The safety is trying a personal assistant for the first time. needs are high priority too as the agent needs to react Need: Esteem (self) and alert law enforcement in case of a breach. For critical Tina: Hey , can you tell me how the and emergency needs (such as requests for ambulance, weather is going to be for the rest of the week? or suicide support) which could result in physical harm, Agent: Sorry, I do not understand. the agent response should be swift and accurate. When Tina: Hey , can you tell me how the needs are of higher-order (love or esteem needs), the the weather is going to be for the rest of the week? user tolerance for system inefficiency is higher. However, Agent: Sorry, I do not understand. the system should still try to maximize the user satisfac- User experience: Tina feels upset that the system tion, be empathetic and polite, and take accountability has failed to recognize her commands because of for unsuccessful sessions. her non-native English accent. It hurts her self- Let us look at some of the user-agent interactions, us- esteem as it is an indirect criticism of her fluency ing hypothetical situations developed based on the needs in English. hierarchy. The agent responses are based on observa- While the example provided above are hypothetical in tions of commercial voice-based personal assistants for nature, our experience interacting with conversational various search tasks. agents are fraught with similar problems. The agent re- • Situation 1: Samantha is driving and wants to find sponses do not follow the norms of human conversation a vegan restaurant near her next stop. She prefers and the user experience is unsatisfactory. As the novelty the restaurant to be rated four star or above. wears off, the user realizes the inability of the system to Need: Physiological (Hunger) fulfill their needs, and therefore, stops using the agent. Samantha: Hey , can you find a Therefore, future systems should be developed with a vegan restaurant near and which is focus on the relative importance of user needs. rated more than four stars? Agent: This is what I found User experience: Samantha found the list hard In this paper, we discussed the design aspects of conversa- to navigate while driving. So she had to stop her tional agents using the lens of human needs. While mod- car and search the restaurant on her phone. It is ern day agents are becoming increasingly humanoid, it is likely that she will never use the agent in future relevant and timely to discuss if the various human-like for a similar task. functionalities are required in these systems. In the first • Situation 2: Kevin wakes up at night and realizes half of the paper, we explored the benefits and drawbacks someone is trying to break into his house. He of some system characteristics (like personality, empathy, needs to contact law enforcement immediately. ethics, voice, embodiment, personalization, and privacy). Need: Safety (Physical and Economic Harm) The interactions between conversational agents and hu- Kevin: Hey , can you call 911? man users are borne out of some need and are therefore, task- or goal-oriented. The user satisfaction is depen- K. Pitsch, L. Schillingmann, et al., Conversational as- dant on the fulfillment of the user objectives, in other sistants for elderly users–the importance of socially words, the success or failure of the tasks. Therefore, we cooperative dialogue, in: Proceedings of the AA- have looked into the hierarchical framework of human MAS Workshop on Intelligent Conversation Agents needs to suggest how an artificially intelligent system in Home and Geriatric Care Applications co-located should assign relative importance to the user tasks. We with the Federated AI Meeting, volume 2338, 2018. discussed how the user needs, the system performance, [4] U. Gnewuch, S. Morana, A. Maedche, Towards and the user satisfaction are directly related to each other.designing cooperative and social conversational Conversational agents – both audio and text-based – can agents for customer service., in: ICIS, 2017. be used for a varied number of tasks, such as searching [5] S. Barko-Sherif, D. Elsweiler, M. Harvey, Conversa- for food or medical help online (physiological need), con- tional agents for recipe recommendation, in: Pro- tacting law enforcement over email, phone, or contact ceedings of the 2020 Conference on Human Infor- forms (safety needs), accessing chat and social media mation Interaction and Retrieval, 2020, pp. 73–82. applications (love and belonging needs), or advancing [6] A. H. Maslow, A theory of human motivation., education (self-actualization). The interface design is Psychological review 50 (1943) 370. important for any human-system interaction, and the [7] T. Bickmore, H. Trinh, R. Asadi, S. Olafsson, Safety success of the conversational agents will depend on their first: conversational agents for health care, in: Stud- usefulness to the user. ies in Conversational UX Design, Springer, 2018, pp. The human needs framework should allow the agent 33–57. to distinguish between various strata of human needs and [8] J. Cassell, T. Bickmore, M. Billinghurst, L. Campbell, their importance. When users interact with the system, K. Chang, H. Vilhjálmsson, H. Yan, Embodiment their utterances can be associated with different levels in conversational interfaces: Rea, in: Proceedings of need. A system should be designed so that the system of the SIGCHI conference on Human Factors in actions – tone, pitch, word choice, urgency, and response Computing Systems, 1999, pp. 520–527. style – is synchronous to the importance of the user [9] J. Cassell, Embodied conversational agents: rep- utterance. We presented some examples of user-agent resentation and intelligence in user interfaces, AI conversation which highlights how the agent response magazine 22 (2001) 67–67. is neutral towards the importance of the user needs. The[10] P. B. Brandtzaeg, A. Følstad, Why people use chat- failure of the agent to assign importance to some criti- bots, in: International conference on internet sci- cal tasks could frustrate users and make them abandon ence, Springer, 2017, pp. 377–392. using the system in future. By leveraging the insights [11] S. Mallios, N. Bourbakis, A survey on human ma- from human psychology – the human needs – the system chine dialogue systems, in: 2016 7th international designers can make future systems more user-friendly conference on information, intelligence, systems & and hence, commercially successful. applications (iisa), IEEE, 2016, pp. 1–7. [12] M. Dubiel, M. Halvey, L. Azzopardi, S. Daronnat, In- One of the limitations of our paper is the theoretical nature of it. While we present some possible use case vestigating how conversational search agents affect scenarios, we would like to analyze some user-system user’s behaviour, performance and search experi- interaction data to see how the user needs influenced ence, in: The second international workshop on the overall experience of the users. Another possible conversational approaches to information retrieval, direction would be to predict the user needs using the 2018. chat transcripts. [13] S. Shiga, H. Joho, R. Blanco, J. R. Trippas, M. Sander- son, Modelling information needs in collaborative search conversations, in: Proceedings of the 40th References international acm sigir conference on research and development in information retrieval, 2017, pp. 715– [1] G. I. Winata, O. Kampman, Y. Yang, A. Dey, P. Fung, 724. Nora the empathetic psychologist., in: INTER- [14] S. Kopp, L. Gesellensetter, N. C. Krämer, SPEECH, 2017, pp. 3437–3438. I. Wachsmuth, A conversational agent as [2] Y.-C. Lee, W.-T. Fu, Supporting peer assessment museum guide–design and evaluation of a real- in education with conversational agents, in: Pro- world application, in: International workshop ceedings of the 24th International Conference on on intelligent virtual agents, Springer, 2005, pp. Intelligent User Interfaces: Companion, 2019, pp. 329–343. 7–8. [15] J. M. Digman, Personality structure: Emergence of [3] S. Kopp, M. Brandt, H. Buschmeier, K. Cyra, the five-factor model, Annual review of psychology F. Freigang, N. Krämer, F. Kummert, C. Opfermann, 41 (1990) 417–440. [16] R. R. McCrae, O. P. John, An introduction to the [29] A. Smith, Cognitive empathy and emotional empa- five-factor model and its applications, Journal of thy in human behavior and evolution, The Psycho- personality 60 (1992) 175–215. logical Record 56 (2006) 3–21. [17] A. Cafaro, H. H. Vilhjálmsson, T. Bickmore, [30] S. D’mello, A. Graesser, Autotutor and affective au- D. Heylen, K. R. Jóhannsdóttir, G. S. Valgarðs- totutor: Learning by talking with cognitively and son, First impressions: Users’ judgments of virtual emotionally intelligent computers that talk back, agents’ personality and interpersonal attitude in ACM Transactions on Interactive Intelligent Sys- first encounters, in: International conference on tems (TiiS) 2 (2013) 1–39. intelligent virtual agents, Springer, 2012, pp. 67–80. [31] R. Looije, M. A. Neerincx, F. Cnossen, Persua- [18] K. Isbister, C. Nass, Consistency of personality in sive robotic assistant for health self-management interactive characters: verbal cues, non-verbal cues, of older adults: Design and evaluation of social be- and user characteristics, International journal of haviors, International Journal of Human-Computer human-computer studies 53 (2000) 251–267. Studies 68 (2010) 386–397. [19] B. Krenn, B. Endrass, F. Kistler, E. André, Effects of [32] O. Perski, D. Crane, E. Beard, J. Brown, Does the ad- language variety on personality perception in em- dition of a supportive chatbot promote user engage- bodied conversational agents, in: International Con- ment with a smoking cessation app? an experimen- ference on Human-Computer Interaction, Springer, tal study, Digital health 5 (2019) 2055207619880676. 2014, pp. 429–439. [33] S. K. D’Mello, B. Lehman, A. Graesser, A motiva- [20] M. Braun, A. Mainz, R. Chadowitz, B. Pfleging, tionally supportive affect-sensitive autotutor, in: F. Alt, At your service: Designing voice assistant New perspectives on affect and learning technolo- personalities to improve automotive user interfaces, gies, Springer, 2011, pp. 113–126. in: Proceedings of the 2019 CHI Conference on Hu- [34] M. Lee, S. Ackermans, N. van As, H. Chang, E. Lu- man Factors in Computing Systems, 2019, pp. 1–11. cas, W. IJsselsteijn, Caring for vincent: a chatbot [21] M. Neff, N. Toothman, R. Bowmani, J. E. F. Tree, for self-compassion, in: Proceedings of the 2019 M. A. Walker, Don’t scratch! self-adaptors reflect CHI Conference on Human Factors in Computing emotional stability, in: International Workshop Systems, 2019, pp. 1–13. on Intelligent Virtual Agents, Springer, 2011, pp. [35] C. Smith, N. Crook, J. Boye, D. Charlton, S. Dobnik, 398–411. D. Pizzi, M. Cavazza, S. Pulman, R. S. De La Camara, [22] S. T. Völkel, R. Schoedel, D. Buschek, C. Stachl, M. Turunen, Interaction strategies for an affective V. Winterhalter, M. Bühner, H. Hussmann, Develop- conversational agent, in: International Conference ing a personality model for speech-based conversa- on Intelligent Virtual Agents, Springer, 2010, pp. tional agents using the psycholexical approach, in: 301–314. Proceedings of the 2020 CHI Conference on Human [36] M. Skowron, H. Pirker, S. Rank, G. Paltoglou, J. Ahn, Factors in Computing Systems, 2020, pp. 1–14. S. Gobron, No peanuts! affective cues for the virtual [23] S. B. Eysenck, H. J. Eysenck, Crime and personality: bartender, in: Twenty-fourth international FLAIRS An empirical study of the three-factor theory, The conference, 2011. British Journal of Criminology 10 (1970) 225–239. [37] M. Smith, Empathy, expansionism, and the ex- [24] A. Danielescu, G. Christian, A bot is not a polyglot: tended mind, Empathy: Philosophical and psy- Designing personalities for multi-lingual conversa- chological perspectives 1 (2011). tional agents, in: Extended Abstracts of the 2018 [38] H. Boukricha, I. Wachsmuth, M. N. Carminati, CHI Conference on Human Factors in Computing P. Knoeferle, A computational model of empathy: Systems, 2018, pp. 1–9. Empirical evaluation, in: 2013 Humaine Associa- [25] H. Kim, D. Y. Koh, G. Lee, J.-M. Park, Y.-k. Lim, tion Conference on Affective Computing and Intel- Designing personalities of conversational agents, ligent Interaction, IEEE, 2013, pp. 1–6. in: Extended Abstracts of the 2019 CHI Conference [39] S. W. McQuiggan, J. L. Robison, R. Phillips, J. C. on Human Factors in Computing Systems, 2019, pp. Lester, Modeling parallel and reactive empathy in 1–6. virtual agents: an inductive approach., in: AAMAS [26] B. F. Rubin, 2017, Alexa, be more human. Inside (1), Citeseer, 2008, pp. 167–174. Amazon’s effort to make its voice assistant smarter, [40] A. Paiva, I. Leite, H. Boukricha, I. Wachsmuth, chattier, and more like you. Retrieved January 05 Empathy in virtual agents and robots: a survey, (2018). ACM Transactions on Interactive Intelligent Sys- [27] G. Fowler, Are smartphones becoming smart alecks, tems (TiiS) 7 (2017) 1–40. Wall Street Journal 15 (2011). [41] S. Wu, B. Keysar, The effect of culture on perspec- [28] S. H. Losoya, N. Eisenberg, Affective empathy. tive taking, Psychological science 18 (2007) 600– (2001). 606. [42] A. C. Elkins, D. C. Derrick, The sound of trust: voice therapist? young people’s ethical perspectives on as a measurement of trust during interactions with the use of fully automated conversational agents embodied conversational agents, Group decision (chatbots) in mental health support, Biomedical and negotiation 22 (2013) 897–913. informatics insights 11 (2019) 1178222619829083. [43] J. F. Nunamaker, D. C. Derrick, A. C. Elkins, J. K. [57] A. Schlesinger, K. P. O’Hara, A. S. Taylor, Let’s talk Burgoon, M. W. Patton, Embodied conversa- about race: Identity, chatbots, and ai, in: Proceed- tional agent-based kiosk for automated interview- ings of the 2018 chi conference on human factors ing, Journal of Management Information Systems in computing systems, 2018, pp. 1–14. 28 (2011) 17–48. [58] A. C. Curry, V. Rieser, # metoo alexa: How conver- [44] W. A. Bainbridge, J. Hart, E. S. Kim, B. Scassellati, sational systems respond to sexual harassment, in: The effect of presence on human-robot interaction, Proceedings of the second acl workshop on ethics in: RO-MAN 2008-The 17th IEEE International Sym- in natural language processing, 2018, pp. 7–14. posium on Robot and Human Interactive Commu- [59] M. Whittaker, K. Crawford, R. Dobbe, G. Fried, nication, IEEE, 2008, pp. 701–706. E. Kaziunas, V. Mathur, S. M. West, R. Richardson, [45] J. Kennedy, P. Baxter, T. Belpaeme, Comparing J. Schultz, O. Schwartz, AI now report 2018, AI Now robot embodiments in a guided discovery learning Institute at New York University New York, 2018. interaction with children, International Journal of [60] C. O’neil, Weapons of math destruction: How big Social Robotics 7 (2015) 293–308. data increases inequality and threatens democracy, [46] M. Rheu, J. Y. Shin, W. Peng, J. Huh-Yoo, Systematic Crown, 2016. review: Trust-building factors and implications for [61] M. Maybury, W. Wahlster, Readings in intelligent conversational agent design, International Journal user interfaces, Morgan Kaufmann, 1998. of Human–Computer Interaction 37 (2021) 81–96. [62] C. A. Thompson, M. H. Goker, P. Langley, A person- [47] S. Brahnam, A. De Angeli, Gender affordances of alized system for conversational recommendations, conversational agents, Interacting with Computers Journal of Artificial Intelligence Research 21 (2004) 24 (2012) 139–153. 393–428. [48] Y. Kim, A. L. Baylor, E. Shen, Pedagogical agents as [63] R. Kocielnik, L. Xiao, D. Avrahami, G. Hsieh, Reflec- learning companions: the impact of agent emotion tion companion: a conversational system for engag- and gender, Journal of Computer Assisted Learning ing users in reflection on physical activity, Proceed- 23 (2007) 220–234. ings of the ACM on Interactive, Mobile, Wearable [49] B. Tay, Y. Jung, T. Park, When stereotypes meet and Ubiquitous Technologies 2 (2018) 1–26. robots: the double-edge sword of robot gender and [64] S. Kaghyan, S. Sarpal, A. Zorilescu, D. Akopian, personality in human–robot interaction, Comput- Review of interactive communication systems for ers in Human Behavior 38 (2014) 75–84. business-to-business (b2b) services, Electronic [50] M. X. Zhou, G. Mark, J. Li, H. Yang, Trusting virtual Imaging 2018 (2018) 117–1. agents: The effect of personality, ACM Transactions [65] W. Cai, J. Grossman, Z. Lin, H. Sheng, J. T.-Z. Wei, on Interactive Intelligent Systems (TiiS) 9 (2019) 1– J. J. Williams, S. Goel, Mathbot: A personalized 36. conversational agent for learning math, Published [51] R. M. Schuetzler, J. S. Giboney, G. M. Grimes, J. F. to ACM (2019). Nunamaker Jr, The influence of conversational [66] F. Cai, S. Wang, M. de Rijke, Behavior-based person- agent embodiment and conversational relevance alization in web search, Journal of the Association on socially desirable responding, Decision Support for Information Science and Technology 68 (2017) Systems 114 (2018) 94–102. 855–868. [52] E. Ruane, A. Birhane, A. Ventresque, Conversa- [67] T. Giorgino, I. Azzini, C. Rognoni, S. Quaglini, tional ai: Social and ethical considerations., in: M. Stefanelli, R. Gretter, D. Falavigna, Automated AICS, 2019, pp. 104–115. spoken dialogue system for hypertensive patient [53] R. Jones, Communication in the real world: An home management, International Journal of Medi- introduction to communication studies, The Saylor cal Informatics 74 (2005) 159–167. Foundation, 2013. [68] H. Tanaka, H. Negoro, H. Iwasaka, S. Nakamura, [54] F. De Saussure, Course in general linguistics, Embodied conversational agents for multimodal au- Columbia University Press, 2011. tomated social skills training in people with autism [55] D. D. Luxton, Ethical implications of conversational spectrum disorders, PloS one 12 (2017) e0182151. agents in global public health, Bulletin of the World [69] K. K. Fitzpatrick, A. Darcy, M. Vierhile, Delivering Health Organization 98 (2020) 285. cognitive behavior therapy to young adults with [56] K. Kretzschmar, H. Tyroll, G. Pavarini, A. Manzini, symptoms of depression and anxiety using a fully I. Singh, N. Y. P. A. Group, Can your phone be your automated conversational agent (woebot): a ran- domized controlled trial, JMIR mental health 4 (2017) e19. [70] H. P. Grice, Logic and conversation, in: Speech acts, Brill, 1975, pp. 41–58. [71] G. Yule, The study of language, Cambridge univer- sity press, 2020. [72] K. Saffarizadeh, M. Boodraj, T. M. Alashoor, et al., Conversational assistants: Investigating privacy concerns, trust, and self-disclosure., in: ICIS, 2017. [73] G. Laban, T. Araujo, The effect of personalization techniques in users’ perceptions of conversational recommender systems, in: Proceedings of the 20th ACM International Conference on Intelligent Vir- tual Agents, 2020, pp. 1–3. [74] S. Ghosh, Designing human-computer conversa- tional systems using needs hierarchy, School of Information Student Research Journal 11 (2021) 3. [75] A. Maslow, K. Lewis, Maslow’s hierarchy of needs, Salenger Incorporated 14 (1987) 987.