The fascinating story of human evolution can be attributed to our ability to speak, write, and communicate complex thoughts. When researchers envision a perfect, artificially intelligent conversational system, they want the system to be human-like. In other words, the system should converse with the same intellect and cognition as humans. Now, the question which we need to ask is if we need a human-like conversational system? Before we engage in the complex endeavor of implementing humanlike characteristics, we should debate if the pursuit of such a system is logical and ethical. We analyze some of the system-level characteristics and discuss their merits and potential of harm. We review some of the latest work on conversational systems to understand how design features are evolving for Conversational Agents. Additionally, we look into the framework of human needs to assess how the system should assign relative importance to user requests, and prioritize user tasks. We draw on the peer work in human-computer interaction, sentiment analysis, and human psychology to provide insights into how future conversational agents should be designed for better user satisfaction.
1. Introduction USD in 2020 to 13.9 billion USD by 20231. Conversational agents are now used by several commercial sectors for As humans, we are fascinated with anything that can rendering support related to healthcare [1], education [2], talk, walk, or behave as humans do. While it is true that elderly care [3], customer service [4], and information any intelligent being should be able to communicate, the retrieval [5]. Some of these systems are voice-based only forms of communication may vary. For a system to in- and also known as intelligent personal assistants (IPAs). teract with humans eficiently, it should speak and write A few popular systems present in the market now are in a manner which is easily understood by the human Amazon’s Alexa2, Google’s Assistant3, Microsoft’s Corusers. In the late 18ℎ century Erasmus Darwin invented tana4, and Apple’s Siri5. a machine that could produce single phonemes and this With the increase of preference for humanoid systems, was probably the first successful attempt of constructing researchers and developers have been increasingly dea machine that could produce human sounds. Around the voted to designing conversational systems which are 1960s, researchers started exploring the idea of a talking more anthropomorphic, or human-like. Human voices – computer. With time, our understanding of science and with options of selecting from multiple speakers, genders, technology developed, and we developed computational and dialects – have replaced robotic voices. To increase systems that can talk and understand natural language. the novelty factor and attractiveness of these systems, Intelligent Personal Assistants (IPAs) have flooded the celebrity voices are also being used. Research attempts market commercially and have become part of our every- are also being made towards more user-friendly and acday lives. We use them on the phone, on smart speakers, cessible user interface, better system-level cognition and and in our cars. It is predicted that the market value response, organic development of natural language diof AI conversational systems will rise from 4.8 billion alogues, and efective ways of presenting the retrieved information. Overall, the above mentioned research directions should help in developing conversational systems which recognizes user sentiment and responds with empathy.
Interaction between users and conversational systems the user, who can explain his information need with long are task- or goal-oriented, and often with a definite set descriptions and more context. of objectives. It could include but not be limited to con- The medium of interaction between the user and the trolling smart home devices – switching on the lights, system influences the design and application of the syssetting an alarm, turning up the temperature. The users tem. A text-based conversational system is referred to also prefer to ‘talk’ to the system, treating it as a human as a chatbot while a voice-based system is called a perconversational partner. This type of user behavior could sonal assistant (or intelligent personal assistant). Use of be encouraged by system novelty or user boredom. Eval- voice enables spoken systems to be used in hands-free uation of such task-based systems are often governed by and eyes-free situation, which is common while driving, the success or failure of the user tasks or fulfillment of the cooking, or working out. All of these situations involve a user objectives, therefore, the system needs to prioritize primary task which is the focus of the user attention. The tasks of higher importance over others. conversation is secondary and is employed to achieve
In this paper, we survey some of the latest papers ex- simple tasks or question answering. A chatbot, however, ploring humanoid features for conversational systems. allows collaboration among multiple users, and presenThe review helps us assess the potential merits and harms tation of lists, images, and videos. Since text allows the of implementing the researched characteristics. Next, we user to scan, the system response can be longer and more use Maslow’s hierarchy of human needs [6] to suggest detailed. A multimodal system like an embodied conhow the system should prioritize between multiple tasks versational agents (ECA) [7, 8, 9] has a virtual face or and assign importance to diferent user needs. Lastly, we body (artificially generated) – in addition to text or voice use some use case scenarios to demonstrate how the ex- – and can therefore, communicate using facial expresisting system may adversely afect the users’ interaction sions, gestures, body language, and non-verbal cues. The experience. ability to display sentiment makes ECAs applicable to
The rest of the paper is organized as follows: In Section mental health domains where the system should be able 2, we review some of the system-level human-like char- to empathize and display emotions. The last decade have acteristics which were implemented in conversational witnessed massive popularity of mobile devices, which agents. In Section 3, we look into Maslow’s Need Hier- has provided a perfect platform for voice-based system. archy and its application in the design of conversational Conversational systems have already found application systems capable of prioritizing user tasks. Lastly, we in searching [10, 11], flight booking services [ 12], and present use case scenarios to highlight some of the issues vacation planning [13, 14]. To address loneliness in pawith existing systems and how the need hierarchy could tients, conversational systems have also been deployed be utilized to mitigate them. In Section 4, we conclude as conversation partners [1]. However, modern day conthe work and propose future directions. versational systems are still in a developing stage and more research is required before mimicking the complex nature of human-human conversations. In the following 2. Exploration of System-level subsections, we look into some of the design aspects (or Characteristics characteristics) introduced by researchers for diferent types of conversational systems.
The popularity of conversational systems – where the conversation could be voice-based or text-based – can be largely attributed to their ability to understand and generate natural language dialogues. A successful design is one which seamlessly integrates with the environment and the system is almost invisible to the user. A major application of conversational systems is in information retrieval, where the user can approach the system with his query, and the system responds with the useful information. However, the user-system interaction in search systems is fraught with problems. First, the user has to represent his information need using a set of keywords (queries). Moreover, towards the beginning of the search session, the user is not cognizant of the exact nature of his information problem. The problem of cold start has been researched in the information community and the cognitive load placed on the user is far from ideal. Use of natural language should reduce the cognitive load of
Personality can be defined as a set of characteristics that determine how a person behaves or reacts to their environment. For characterizing human personalities, researchers often use the Big Five Model or the OCEAN model. The OCEAN [15, 16] is an acronym composed of five diferent personality types: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. Multiple studies [17, 18, 19] have used this model to explain how personalities should be designed for conversational agents.
Braun et al. (2019) [20] suggested that the user wants the agent to reflect the users’ personality. Other studies found that extraversion is the most common user personality, which was also found in conversational agents. Nef et al. (2011) [ 21] reported that the users sensed neuroticism in the CA. However, many researchers feel that the OCEAN model is neither suficient nor appropriate to model CA personalities and have proposed an alternate model of personalities [22] – the Three-Factor Model [23] – which is quite diferent from the OCEAN. Another study [24] discusses how the language used by conversational agents can influence the way users perceive their personality. The locale and cultural practices also play an important role in how the users perceive the agent. Even slight variations of tone and acoustics can play an essential role in user perception. Kim et al. (2019) [25] categorized personality traits into Common Trait, Distinctive Trait, and Neutral Trait. The channels of expression (of personality) were also divided into three categories: Ways of Providing Service, Types of Service, and Language & Appearance. The more popular and ubiquitous conversational agents like Alexa [26] were designed to be smart, approachable, humble, enthusiastic, and helpful. In contrast, Siri was designed to be friendly and humble, but with an edge to her personality [27].
tional agents, it makes the conversation more natural.
Voice-based systems allow the users to multitask while performing a primary task (driving or cooking). Spoken dialogues are the natural form of communication and promote trust between the participants [42]. User comfort and satisfaction increase significantly when the user can trust the agent. Since emotion is an important aspect of the human-human conversation, and facial, bodily, or gestural expressions are not always available, the importance of voice is magnified in a user-agent interaction.
Nunamaker et al. (2011) [43] reported that users perceive certain gender to be more trustworthy, able, or likable.
Therefore, in a voice-based environment, the choice of voice (male or female), the pitch, and the loudness can afect the user’s perception of the agent. Also, by utilizing the acoustic and prosodic properties of voice, the agent can identify user emotions and express its feelings.
Danielescu and Christian (2018) [24] found that users want more control and would prefer to select the type of voice of the agent.
In any conversation, emotions indicate the level of engagement and satisfaction. Therefore, if we consider 2.4. Embodiment individual utterances, the emotions displayed should be considered while generating the agent response. Simi- In addition to voice, conversational agents can also have larly, the emotion expressed in the agent response can in- an artificially generated face or body. Research on emlfuence the user engaged in the discourse. Empathy is an bodied agents suggests that a body and voice can help essential socio-emotional behavior for efective interper- users to socially accept an agent. The presence of versonal communication. During a conversation, humans bal and non-verbal cues in embodied agents allows for often exhibit afective empathy [ 28] which can be de- the expression of empathy [25] and emotions. Embodied ifned as the human nature of automatic and unconscious agents – using multimodal channels – are perceived to mimicking of the other participants to match or mirror be more socially present for the users [44, 45]. Some their emotions. On the other hand, for cognitive empathy studies [8, 9] claim that human-like intelligence can only [29], we consider the perspective or mental state of the be exhibited by artificially intelligent systems through conversational partner before reacting. A general ten- non-verbal cues, and that is only possible when using an dency of researchers is to create conversational agents embodied conversational agent. Rheu et al. (2021) [46] which are sympathetic [30, 31], supportive [32, 33] or suggested that embodiment can make the agent more compassionate [34]. Few agents exhibit qualities like af- trustworthy. fective matching [35, 36] and mirroring mechanism [37]. Despite the afordances ofered by embodied converComplex models like the EMMA framework [38] and sational agents, we must look at the potential of harm. the CARE framework [39] have also been developed for Gender of the embodiment has also been a topic of disagents. However, if we look at the existing state-of-the- cussion among researchers. While some researchers art assistants, they are not empathetic. Since empathy think that androgynous personas will contribute toward consists of multiple layers [40], the implementation of unbiased agents, others argue that humanizing agents complex empathetic expressions become extremely chal- will lead to better performance. User interactions with lenging. Integrating human-like empathetic responses in female embodied agents resulted in more sexual and a conversational system could be thought of as a three- swear words [47]. The efect of gender was also oblevel process: perspective-taking, context generation, and served in other studies where the users perpetuated expression. Perspective-taking [41] means understand- gender stereotypes specific to agent personality and ing the views, beliefs, desires, and intentions of the user. roles [48, 49, 50, 51]. While embodied agents can be designed with the option to stop any gender manifestations [52], it can also alienate the user from the agent and lead to fewer interactions. 2.5. Ethics the dialogue style [67] and voice parameters could be adjusted to meet the user’s preferences.
In any type of discourse, language is a primary compo- Personalization eforts can be grouped under two nent that reflects the political, sociological, and cultural broad categories. The agent has to either store inforconditions of a particular time [53]. The choice and usage mation from every interactive session (implicit personalof words are governed by the context, and the synchronic ization) [68] or ask the user a set of questions (explicit nature of language [54]. A word that is deemed accept- personalization) [69] at the beginning of every session. able at present may not be acceptable in the future. If The two approaches present a trade-of between convewe look at the word ‘awful,’ it has a negative meaning nience and privacy. associated with it. However, it originated as a shortened One important aspect of personalization is the resform for “full of awe,” which is a positive phrase (refer- olution of conversational implicatures in human-agent ring to something that inspires wonder). Therefore, a dialogues. Conversational implicature is an important linconversational agent cannot focus only on the linguistic guistic phenomenon that allows humans to imply meanaspects and ignore the socio-cultural contexts. ings without clarifying them explicitly [70]. It helps to
Ethics depends on four major factors: time, context, keep the conversation short and hedge negative emouser perception, and user’s socio-cultural aspects. Ethics tions. Such implications are also common for users who in conversation involves knowing which words to use are depressed or suicidal [7]. Yule (2020) [71] shows how in a dialogue and which words to avoid. Modern-day complex implications are dificult to understand, even in conversational agents are deployed in the field of mental human-human conversations. Since existing agents lack health [55], where they talk to people and make them suficient cognition to interpret implications in human feel better. Since people with mental health issues are dialogues, they must ask clarifying questions and resolve vulnerable, the conversation should be carefully struc- ambiguities. Such clarifications increase the number of tured to avoid hurting anyone’s sentiments. Kretzschmar turns and may lower user engagement. One possible et al. (2019) [56] discussed that such agents do not always solution is to save dialogues from previous interactions consider the potential of harm. with the user.
Schlesinger et al. (2018) [57] found that some agents However, while personalization and implicature resouse a collection of blacklisted words to detect undesir- lution will lead to better user experience, it comes at the able speech and, therefore, deflect questions related to cost of reduced privacy. Interactions with conversational race. However, users often perceive this deflection as agents – using natural free-form language – can lead an endorsement of racial hate or nonchalance toward to the disclosure of personal and sensitive information racial issues. When the user initiates an open-domain related to health, security, or finance. Safarizadeh et al. conversation on sexual harassment, some agents [58] (2017) [72] believes that more users prefer privacy over even responded with counter-aggression or flirtatious be- personalized response. An acceptable solution would havior. While such behavior can be attributed to training be to allow the users to decide how much personal indata, the developers cannot ignore the lack of ethics in ex- formation they want to share and their desired privacy isting conversational agents. Whittaker et al. (2018) [59] levels. The agent should disclose all the signals it has and O’neil (2016) [60] argue that the developers often collected from the user, both implicitly and explicitly. ignore ethical considerations in favor of technical aspects. Also, the collected information should be encrypted to Conversational agents must be evaluated continuously or prevent unauthorized access. Should the user exercise periodically to confirm that the systems are not behaving the right to be forgotten, the agent must clear all the unethically. To use such agents in everyday life – where user’s stored data. While this would alleviate privacy they interact with humans – the potential harm must be concerns, it adversely afects the personalization eforts. mitigated. Any attempts by the agent to collect data without the user’s consent could be perceived as a threat [73] and
After an extensive exploration of existing literature on intelligent conversational agents, we identified one major shortcoming in current state-of-the-art systems. The inability of the agents to prioritize tasks is a major challenge for intelligent conversational agents. Therefore, in the following section, we propose a framework that can be used to prioritize user tasks.
Increasing personalization allows the agent to behave uniquely for every user, tailoring the agent’s decisions to the personality and preferences of the user. Personalization enables the agent to dynamically adapt to the user and make better recommendations, which increases user satisfaction [61, 62, 63]. Several studies have discussed how personalized agents can be more efective in healthcare [63], libraries [61], business [64] and education [65]. The relevancy of results could be improved [66], and
Our review of existing conversational systems highlighted that although several intelligent functionalities have been proposed and implemented in existing systems, the system design is still not ideal for context mediated behavior and task prioritization. Prioritizing tasks - by intelligent agents - is essential to guarantee a faster turnaround time with greater accuracy for tasks of higher priority which can ensure better user satisfaction. There exists a strong relationship between user satisfaction and the design of conversational systems. In order to widen the scope and application of these systems, it is essential to look into the human aspect of such systems in addition to the computational side. For example, insights from human psychology [74] – such as the framework of human needs – could significantly improve the operationalization and functioning of conversational systems.
The development and design of any system is borne out of need to solve a problem, or to improve an existing solution. Therefore, any system design should concentrate on the needs of the end user and the potential of the system to satisfy those needs, either working on its own or in collaboration with the user.
Maslow [75] looked into the story of human evolution and proposed a hierarchical framework to explain how diferent needs are prioritized by the human mind. The human mind is motivated by the instincts to survive, both as an individual and as a species. Therefore, it assigns varying levels of importance to the things around us. The level of satisfaction is higher if a higher order need is satisfied. For example, any primitive organism aims to secure the basic items which it needs to survive. This could include food, water, air, and temperature optimal for growth. This is no diferent for a human baby. Any potential threat to survival is met with the desire to fight or flight. As an organism evolves – evolution of life or a baby growing into an adult – the basic needs are supplemented by higher order needs. Such needs – which could be philanthropic, spiritual, or materialistic – are not replacements for basic needs. The fundamental needs are still important for survival but the higher order needs go beyond the needs of the self.
The tier-based structure of Maslow’s Need Hierarchy contains five levels and is shown in Figure 1. The diferent levels in the hierarchy suggests that before an individual pursues any top-level needs, he must ensure that the fundamental needs (those related to survival) are satisfied. The two levels at the bottom represents psychological and safety needs which are essential for survival. This is followed by two more levels of psychological needs (love and esteem). Finally, we see the need self-actualization at the top. We explain these five levels in detail, along with the three new levels which were added later. • Physiological Needs:
The most primitive needs for any living organism (including humans) are to survive and grow. The requirements for sustaining life include food, water, air, and sleep. Once survival is ensured, the focus shifts to maintaining optimal conditions for growth. • Safety Needs:
Safety requirements are the second most important need for humans. This means safety for self and those who are closest to them. Safety is often connected to a known order. Therefore, any unpredictability or course of events which could pose a threat to life, or living conditions of an individual, is detrimental to the user experience and satisfaction. • Community and Belonging:
For a majority of humans, their existence is not merely as an individual but as a part of a larger social group or community. Humans cherish the emotional connection and togetherness which they feel with their family, friends, colleagues, and romantic partners. Community and belonging needs allow humans to avoid loneliness and leads to psychological well-being. The transition from individual survival to community needs occurs only when there is no threat to the underlying need levels. • Esteem Needs:
Esteem needs are related to the feeling of selfworth and can be broadly categorized into two categories: self-respect and respect from peers and community. Humans possess a strong desire to be accepted, appreciated, and validated by their social circles. They also value freedom and choice,
and feeling confident and competent. A system and fulfilled from bottom upwards. Therefore, only when should respect the esteem needs of the users as the primary needs like hunger and thirst are satisfied, any action detrimental to esteem needs will likely does the individual look for “higher” needs. There is also lower the satisfaction levels of the user. an inherent relationship between human motivation and • Cognitive Needs: needs. Maslow suggested that for the fundamental or While cognitive needs were not part of the ini- basic needs (physiological and safety needs like air, water, tial need hierarchy, they were later added to the food, shelter), as the deficiency increases, the motivation original five levels. These needs are the dominant increases as well. Therefore, these needs as strongest moreason why humans strive to acquire more knowl- tivators for any human being. However, the fulfillment edge and challenge their intellect by partaking in of these needs results in decrease in motivation. A man cognitively complex tasks. who has suficient bread to eat does not strive for more • Aesthetic Needs: bread. Instead, they look for higher-order needs (love, Aesthetic needs were appended to the original esteem and self-actualization). However, fulfillment of need hierarchy. These needs are symbolic of higher needs does not lower motivation. Instead, motivathe human fascination to create and appreciate tion keeps increasing as these needs are being met. This beauty, artistic marvels. explains why humans strive for more fame, money, and • Self-Actualization: achievements, although they have enough.
In the original five-tiered need hierarchy, self- While the framework of human needs proposed by actualization was at the top of the hierarchical Maslow is hierarchical, researchers have debated if the pyramid, which means that the fulfillment of diferent levels are mutually exclusive. Also, how often this need is prioritized after all the lower level are the diferent levels pursued simultaneously? When needs are met. Self-actualization is the urge to there is no food or water (‘bread’ as Maslow calls it), reach one’s true potential and fulfill one’s talents. and hunger is the motivating factor, a man will prioritize The self-actualization needs could be fulfilled by the need for food above others. But that does not stop gaining knowledge, receiving awards in one’s do- him from looking for a safe shelter, obtaining education, main of work, and fulfilling long- and short-term or looking for a better job. While the need levels may dreams. The achievements of goals by the indi- be clearly distinguishable from each other, the actions vidual are metrics to judge how well the needs may not be. Certain actions may fulfill the lower-level were met. needs in the short-term and higher-levels in the longer • Self-Transcendence: run. However, it can be agreed upon that social, cultural, Like cognitive and aesthetic needs, self- and economic aspects (which are specific to every inditranscendence was added later to the needs vidual) governs how a human prioritizes the diferent hierarchy. These needs are mostly spiritual in needs. Deficiency in a lower-order need may act as a nature and connects the individual to a higher deterrent to pursue higher needs, but there are many purpose or entity. Spiritual needs, which may exceptions. Many high-achieving individuals, the basic or may not be connected to organized religion, needs are sacrificed to fulfill esteem and self-actualization transcend the materialistic pleasures and gives needs. For others, the needs are in a state of constant meaning to the life of individuals. change throughout the life of the individual. The basic needs (in infants) are supplemented by safety and love needs (as they mature), and esteem and self-actualization It is quite true that man lives by bread needs (when adults). Certain life situations (financial alone — when there is no bread. But what hardship, health complications) may motivate some of happens to man’s desires when there is the needs more than others but for every individual, there plenty of bread and when his belly is is a unique balance between the diferent needs. The chronically filled? At once other (and needs framework, while being hierarchical, has varying “higher”) needs emerge and these, rather amounts of overlap depending on the individual. than physiological hungers, dominate the As we discussed in Section 2, many advanced features organism. And when these in turn are sat- have been implemented in current conversational agents. isfied, again new (and still “higher”) needs Surprisingly, none of these features help to understand emerge and so on. This is what we mean the context - from user utterance - and prioritize tasks by saying that the basic human needs are accordingly. For example, the acoustic properties of user organized into a hierarchy of relative pre- utterance changes with the user’s mood and situation. potency. (Maslow, 1943, p. 375) [6] Therefore, an intelligent agent should use such features to determine the context of the task. Subsequently, the contextual information can be utilized to decide on task
teristics which the users desire of conversational agents. While each of those functionalities will require careful development (so as to avoid any potential for harm), the responses and urgency of the agent should consider the relative importance of human needs. Our exploration of Maslow’s framework [6] suggested the user have limited patience when their basic needs are threatened. The range of use cases could vary from an user looking for shelter homes, community food kitchens to another looking for a nearby restaurant. The physiological needs have varying degrees of importance and the users need the agent to be empathetic to their problem. The safety needs are high priority too as the agent needs to react and alert law enforcement in case of a breach. For critical and emergency needs (such as requests for ambulance, or suicide support) which could result in physical harm, the agent response should be swift and accurate. When the needs are of higher-order (love or esteem needs), the user tolerance for system ineficiency is higher. However, the system should still try to maximize the user satisfaction, be empathetic and polite, and take accountability for unsuccessful sessions.
Let us look at some of the user-agent interactions, using hypothetical situations developed based on the needs hierarchy. The agent responses are based on observations of commercial voice-based personal assistants for various search tasks.
• Situation 1: Samantha is driving and wants to find a vegan restaurant near her next stop. She prefers the restaurant to be rated four star or above.
Need: Physiological (Hunger) Samantha: Hey <agent-name>, can you find a vegan restaurant near <city-name> and which is rated more than four stars? Agent: This is what I found <list of restaurants which is read out loud> User experience: Samantha found the list hard to navigate while driving. So she had to stop her car and search the restaurant on her phone. It is likely that she will never use the agent in future for a similar task. • Situation 2: Kevin wakes up at night and realizes someone is trying to break into his house. He needs to contact law enforcement immediately. Need: Safety (Physical and Economic Harm) Kevin: Hey <agent-name>, can you call 911?
Agent: Sorry, I do not understand.
User experience: Kevin realized that the agent is failing to recognize his panicked voice. He found his phone to call law enforcement. • Situation 3: Tarek is lonely and struggling with health issues. He decides to talk to the agent about his health condition.
Need: Love and Belonging (Emotional Support) Kevin: Hey <agent-name>, I am having trouble with <starts to explain his medical problem> Agent: <cuts him of> This is what I found.
User experience: Tarek is upset because the agent not only failed to maintain conversation but interrupted him and provided irrelevant information. There is strong probability that he will not use the agent in the future. • Situation 4: Tina is not a native speaker of English but prides herself in being fluent in English. She is trying a personal assistant for the first time.
Need: Esteem (self) Tina: Hey <agent-name>, can you tell me how the weather is going to be for the rest of the week? Agent: Sorry, I do not understand.
Tina: Hey <agent-name>, can you tell me how the weather is going to be for the rest of the week? Agent: Sorry, I do not understand.
User experience: Tina feels upset that the system has failed to recognize her commands because of her non-native English accent. It hurts her selfesteem as it is an indirect criticism of her fluency in English.
While the example provided above are hypothetical in nature, our experience interacting with conversational agents are fraught with similar problems. The agent responses do not follow the norms of human conversation and the user experience is unsatisfactory. As the novelty wears of, the user realizes the inability of the system to fulfill their needs, and therefore, stops using the agent. Therefore, future systems should be developed with a focus on the relative importance of user needs.
In this paper, we discussed the design aspects of conversational agents using the lens of human needs. While modern day agents are becoming increasingly humanoid, it is relevant and timely to discuss if the various human-like functionalities are required in these systems. In the first half of the paper, we explored the benefits and drawbacks of some system characteristics (like personality, empathy, ethics, voice, embodiment, personalization, and privacy).
The interactions between conversational agents and human users are borne out of some need and are therefore, task- or goal-oriented. The user satisfaction is depen- K. Pitsch, L. Schillingmann, et al., Conversational asdant on the fulfillment of the user objectives, in other sistants for elderly users–the importance of socially words, the success or failure of the tasks. Therefore, we cooperative dialogue, in: Proceedings of the AAhave looked into the hierarchical framework of human MAS Workshop on Intelligent Conversation Agents needs to suggest how an artificially intelligent system in Home and Geriatric Care Applications co-located should assign relative importance to the user tasks. We with the Federated AI Meeting, volume 2338, 2018. discussed how the user needs, the system performance, [4] U. Gnewuch, S. Morana, A. Maedche, Towards and the user satisfaction are directly related to each other. designing cooperative and social conversational Conversational agents – both audio and text-based – can agents for customer service., in: ICIS, 2017. be used for a varied number of tasks, such as searching [5] S. Barko-Sherif, D. Elsweiler, M. Harvey, Conversafor food or medical help online (physiological need), con- tional agents for recipe recommendation, in: Protacting law enforcement over email, phone, or contact ceedings of the 2020 Conference on Human Inforforms (safety needs), accessing chat and social media mation Interaction and Retrieval, 2020, pp. 73–82. applications (love and belonging needs), or advancing [6] A. H. Maslow, A theory of human motivation., education (self-actualization). The interface design is Psychological review 50 (1943) 370. important for any human-system interaction, and the [7] T. Bickmore, H. Trinh, R. Asadi, S. Olafsson, Safety success of the conversational agents will depend on their ifrst: conversational agents for health care, in: Studusefulness to the user. ies in Conversational UX Design, Springer, 2018, pp.
The human needs framework should allow the agent 33–57. to distinguish between various strata of human needs and [8] J. Cassell, T. Bickmore, M. Billinghurst, L. Campbell, their importance. When users interact with the system, K. Chang, H. Vilhjálmsson, H. Yan, Embodiment their utterances can be associated with diferent levels in conversational interfaces: Rea, in: Proceedings of need. A system should be designed so that the system of the SIGCHI conference on Human Factors in actions – tone, pitch, word choice, urgency, and response Computing Systems, 1999, pp. 520–527. style – is synchronous to the importance of the user [9] J. Cassell, Embodied conversational agents: reputterance. We presented some examples of user-agent resentation and intelligence in user interfaces, AI conversation which highlights how the agent response magazine 22 (2001) 67–67. is neutral towards the importance of the user needs. The [10] P. B. Brandtzaeg, A. Følstad, Why people use chatfailure of the agent to assign importance to some criti- bots, in: International conference on internet scical tasks could frustrate users and make them abandon ence, Springer, 2017, pp. 377–392. using the system in future. By leveraging the insights [11] S. Mallios, N. Bourbakis, A survey on human mafrom human psychology – the human needs – the system chine dialogue systems, in: 2016 7th international designers can make future systems more user-friendly conference on information, intelligence, systems & and hence, commercially successful. applications (iisa), IEEE, 2016, pp. 1–7.
One of the limitations of our paper is the theoretical [12] M. Dubiel, M. Halvey, L. Azzopardi, S. Daronnat, Innature of it. While we present some possible use case vestigating how conversational search agents afect scenarios, we would like to analyze some user-system user’s behaviour, performance and search experiinteraction data to see how the user needs influenced ence, in: The second international workshop on the overall experience of the users. Another possible conversational approaches to information retrieval, direction would be to predict the user needs using the 2018. chat transcripts. [13] S. Shiga, H. Joho, R. Blanco, J. R. Trippas, M. Sanderson, Modelling information needs in collaborative search conversations, in: Proceedings of the 40th References international acm sigir conference on research and development in information retrieval, 2017, pp. 715– [1] G. I. Winata, O. Kampman, Y. Yang, A. Dey, P. Fung, 724.
Nora the empathetic psychologist., in: INTER- [14] S. Kopp, L. Gesellensetter, N. C. Krämer, SPEECH, 2017, pp. 3437–3438. I. Wachsmuth, A conversational agent as [2] Y.-C. Lee, W.-T. Fu, Supporting peer assessment museum guide–design and evaluation of a realin education with conversational agents, in: Pro- world application, in: International workshop ceedings of the 24th International Conference on on intelligent virtual agents, Springer, 2005, pp. Intelligent User Interfaces: Companion, 2019, pp. 329–343.
7–8. [15] J. M. Digman, Personality structure: Emergence of [3] S. Kopp, M. Brandt, H. Buschmeier, K. Cyra, the five-factor model, Annual review of psychology
F. Freigang, N. Krämer, F. Kummert, C. Opfermann, 41 (1990) 417–440. [16] R. R. McCrae, O. P. John, An introduction to the [29] A. Smith, Cognitive empathy and emotional empaifve-factor model and its applications, Journal of thy in human behavior and evolution, The Psychopersonality 60 (1992) 175–215. logical Record 56 (2006) 3–21. [17] A. Cafaro, H. H. Vilhjálmsson, T. Bickmore, [30] S. D’mello, A. Graesser, Autotutor and afective auD. Heylen, K. R. Jóhannsdóttir, G. S. Valgarðs- totutor: Learning by talking with cognitively and son, First impressions: Users’ judgments of virtual emotionally intelligent computers that talk back, agents’ personality and interpersonal attitude in ACM Transactions on Interactive Intelligent Sysifrst encounters, in: International conference on tems (TiiS) 2 (2013) 1–39.
intelligent virtual agents, Springer, 2012, pp. 67–80. [31] R. Looije, M. A. Neerincx, F. Cnossen, Persua[18] K. Isbister, C. Nass, Consistency of personality in sive robotic assistant for health self-management interactive characters: verbal cues, non-verbal cues, of older adults: Design and evaluation of social beand user characteristics, International journal of haviors, International Journal of Human-Computer human-computer studies 53 (2000) 251–267. Studies 68 (2010) 386–397. [19] B. Krenn, B. Endrass, F. Kistler, E. André, Efects of [32] O. Perski, D. Crane, E. Beard, J. Brown, Does the adlanguage variety on personality perception in em- dition of a supportive chatbot promote user engagebodied conversational agents, in: International Con- ment with a smoking cessation app? an experimenference on Human-Computer Interaction, Springer, tal study, Digital health 5 (2019) 2055207619880676. 2014, pp. 429–439. [33] S. K. D’Mello, B. Lehman, A. Graesser, A motiva[20] M. Braun, A. Mainz, R. Chadowitz, B. Pfleging, tionally supportive afect-sensitive autotutor, in: F. Alt, At your service: Designing voice assistant New perspectives on afect and learning technolopersonalities to improve automotive user interfaces, gies, Springer, 2011, pp. 113–126. in: Proceedings of the 2019 CHI Conference on Hu- [34] M. Lee, S. Ackermans, N. van As, H. Chang, E. Luman Factors in Computing Systems, 2019, pp. 1–11. cas, W. IJsselsteijn, Caring for vincent: a chatbot [21] M. Nef, N. Toothman, R. Bowmani, J. E. F. Tree, for self-compassion, in: Proceedings of the 2019 M. A. Walker, Don’t scratch! self-adaptors reflect CHI Conference on Human Factors in Computing emotional stability, in: International Workshop Systems, 2019, pp. 1–13. on Intelligent Virtual Agents, Springer, 2011, pp. [35] C. Smith, N. Crook, J. Boye, D. Charlton, S. Dobnik, 398–411. D. Pizzi, M. Cavazza, S. Pulman, R. S. De La Camara, [22] S. T. Völkel, R. Schoedel, D. Buschek, C. Stachl, M. Turunen, Interaction strategies for an afective V. Winterhalter, M. Bühner, H. Hussmann, Develop- conversational agent, in: International Conference ing a personality model for speech-based conversa- on Intelligent Virtual Agents, Springer, 2010, pp. tional agents using the psycholexical approach, in: 301–314.
Proceedings of the 2020 CHI Conference on Human [36] M. Skowron, H. Pirker, S. Rank, G. Paltoglou, J. Ahn, Factors in Computing Systems, 2020, pp. 1–14. S. Gobron, No peanuts! afective cues for the virtual [23] S. B. Eysenck, H. J. Eysenck, Crime and personality: bartender, in: Twenty-fourth international FLAIRS An empirical study of the three-factor theory, The conference, 2011.
British Journal of Criminology 10 (1970) 225–239. [37] M. Smith, Empathy, expansionism, and the ex[24] A. Danielescu, G. Christian, A bot is not a polyglot: tended mind, Empathy: Philosophical and psyDesigning personalities for multi-lingual conversa- chological perspectives 1 (2011). tional agents, in: Extended Abstracts of the 2018 [38] H. Boukricha, I. Wachsmuth, M. N. Carminati, CHI Conference on Human Factors in Computing P. Knoeferle, A computational model of empathy: Systems, 2018, pp. 1–9. Empirical evaluation, in: 2013 Humaine Associa[25] H. Kim, D. Y. Koh, G. Lee, J.-M. Park, Y.-k. Lim, tion Conference on Afective Computing and IntelDesigning personalities of conversational agents, ligent Interaction, IEEE, 2013, pp. 1–6. in: Extended Abstracts of the 2019 CHI Conference [39] S. W. McQuiggan, J. L. Robison, R. Phillips, J. C. on Human Factors in Computing Systems, 2019, pp. Lester, Modeling parallel and reactive empathy in 1–6. virtual agents: an inductive approach., in: AAMAS [26] B. F. Rubin, 2017, Alexa, be more human. Inside (1), Citeseer, 2008, pp. 167–174.
Amazon’s efort to make its voice assistant smarter, [40] A. Paiva, I. Leite, H. Boukricha, I. Wachsmuth, chattier, and more like you. Retrieved January 05 Empathy in virtual agents and robots: a survey, (2018). ACM Transactions on Interactive Intelligent Sys[27] G. Fowler, Are smartphones becoming smart alecks, tems (TiiS) 7 (2017) 1–40.
Wall Street Journal 15 (2011). [41] S. Wu, B. Keysar, The efect of culture on perspec[28] S. H. Losoya, N. Eisenberg, Afective empathy. tive taking, Psychological science 18 (2007) 600– (2001). 606. [42] A. C. Elkins, D. C. Derrick, The sound of trust: voice therapist? young people’s ethical perspectives on as a measurement of trust during interactions with the use of fully automated conversational agents embodied conversational agents, Group decision (chatbots) in mental health support, Biomedical and negotiation 22 (2013) 897–913. informatics insights 11 (2019) 1178222619829083. [43] J. F. Nunamaker, D. C. Derrick, A. C. Elkins, J. K. [57] A. Schlesinger, K. P. O’Hara, A. S. Taylor, Let’s talk Burgoon, M. W. Patton, Embodied conversa- about race: Identity, chatbots, and ai, in: Proceedtional agent-based kiosk for automated interview- ings of the 2018 chi conference on human factors ing, Journal of Management Information Systems in computing systems, 2018, pp. 1–14. 28 (2011) 17–48. [58] A. C. Curry, V. Rieser, # metoo alexa: How conver[44] W. A. Bainbridge, J. Hart, E. S. Kim, B. Scassellati, sational systems respond to sexual harassment, in: The efect of presence on human-robot interaction, Proceedings of the second acl workshop on ethics in: RO-MAN 2008-The 17th IEEE International Sym- in natural language processing, 2018, pp. 7–14. posium on Robot and Human Interactive Commu- [59] M. Whittaker, K. Crawford, R. Dobbe, G. Fried, nication, IEEE, 2008, pp. 701–706. E. Kaziunas, V. Mathur, S. M. West, R. Richardson, [45] J. Kennedy, P. Baxter, T. Belpaeme, Comparing J. Schultz, O. Schwartz, AI now report 2018, AI Now robot embodiments in a guided discovery learning Institute at New York University New York, 2018. interaction with children, International Journal of [60] C. O’neil, Weapons of math destruction: How big Social Robotics 7 (2015) 293–308. data increases inequality and threatens democracy, [46] M. Rheu, J. Y. Shin, W. Peng, J. Huh-Yoo, Systematic Crown, 2016.
review: Trust-building factors and implications for [61] M. Maybury, W. Wahlster, Readings in intelligent conversational agent design, International Journal user interfaces, Morgan Kaufmann, 1998. of Human–Computer Interaction 37 (2021) 81–96. [62] C. A. Thompson, M. H. Goker, P. Langley, A person[47] S. Brahnam, A. De Angeli, Gender afordances of alized system for conversational recommendations, conversational agents, Interacting with Computers Journal of Artificial Intelligence Research 21 (2004) 24 (2012) 139–153. 393–428. [48] Y. Kim, A. L. Baylor, E. Shen, Pedagogical agents as [63] R. Kocielnik, L. Xiao, D. Avrahami, G. Hsieh, Refleclearning companions: the impact of agent emotion tion companion: a conversational system for engagand gender, Journal of Computer Assisted Learning ing users in reflection on physical activity, Proceed23 (2007) 220–234. ings of the ACM on Interactive, Mobile, Wearable [49] B. Tay, Y. Jung, T. Park, When stereotypes meet and Ubiquitous Technologies 2 (2018) 1–26. robots: the double-edge sword of robot gender and [64] S. Kaghyan, S. Sarpal, A. Zorilescu, D. Akopian, personality in human–robot interaction, Comput- Review of interactive communication systems for ers in Human Behavior 38 (2014) 75–84. business-to-business (b2b) services, Electronic [50] M. X. Zhou, G. Mark, J. Li, H. Yang, Trusting virtual Imaging 2018 (2018) 117–1.
agents: The efect of personality, ACM Transactions [65] W. Cai, J. Grossman, Z. Lin, H. Sheng, J. T.-Z. Wei, on Interactive Intelligent Systems (TiiS) 9 (2019) 1– J. J. Williams, S. Goel, Mathbot: A personalized 36. conversational agent for learning math, Published [51] R. M. Schuetzler, J. S. Giboney, G. M. Grimes, J. F. to ACM (2019).
Nunamaker Jr, The influence of conversational [66] F. Cai, S. Wang, M. de Rijke, Behavior-based personagent embodiment and conversational relevance alization in web search, Journal of the Association on socially desirable responding, Decision Support for Information Science and Technology 68 (2017) Systems 114 (2018) 94–102. 855–868. [52] E. Ruane, A. Birhane, A. Ventresque, Conversa- [67] T. Giorgino, I. Azzini, C. Rognoni, S. Quaglini, tional ai: Social and ethical considerations., in: M. Stefanelli, R. Gretter, D. Falavigna, Automated AICS, 2019, pp. 104–115. spoken dialogue system for hypertensive patient [53] R. Jones, Communication in the real world: An home management, International Journal of Mediintroduction to communication studies, The Saylor cal Informatics 74 (2005) 159–167.
Foundation, 2013. [68] H. Tanaka, H. Negoro, H. Iwasaka, S. Nakamura, [54] F. De Saussure, Course in general linguistics, Embodied conversational agents for multimodal au
Columbia University Press, 2011. tomated social skills training in people with autism [55] D. D. Luxton, Ethical implications of conversational spectrum disorders, PloS one 12 (2017) e0182151. agents in global public health, Bulletin of the World [69] K. K. Fitzpatrick, A. Darcy, M. Vierhile, Delivering Health Organization 98 (2020) 285. cognitive behavior therapy to young adults with [56] K. Kretzschmar, H. Tyroll, G. Pavarini, A. Manzini, symptoms of depression and anxiety using a fully I. Singh, N. Y. P. A. Group, Can your phone be your automated conversational agent (woebot): a randomized controlled trial, JMIR mental health 4 (2017) e19. [70] H. P. Grice, Logic and conversation, in: Speech
acts, Brill, 1975, pp. 41–58. [71] G. Yule, The study of language, Cambridge univer
sity press, 2020. [72] K. Safarizadeh, M. Boodraj, T. M. Alashoor, et al.,
Conversational assistants: Investigating privacy concerns, trust, and self-disclosure., in: ICIS, 2017. [73] G. Laban, T. Araujo, The efect of personalization techniques in users’ perceptions of conversational recommender systems, in: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, 2020, pp. 1–3. [74] S. Ghosh, Designing human-computer conversational systems using needs hierarchy, School of
Information Student Research Journal 11 (2021) 3. [75] A. Maslow, K. Lewis, Maslow’s hierarchy of needs,
Salenger Incorporated 14 (1987) 987.