So what's the Value of Conversational Agents in E-commerce Retailers? University of Strathclyde , Glasgow G1 1XQ , UK https://

Conversational Agents (CAs) are increasing in popularity very fast. They are used for helping people in many di erent domains. Recently they started to be used in customer service. Many businesses are already using CAs. However, there is a gap on how much value can these CAs bring to a business and why they should use them. Firstly, we need to understand the e ectiveness of current CAs in the industry and identify challenges and limitations that customers are facing. Next we will develop new CAs which will address these challenges and limitations. Finally, we will improve these CAs with new technologies.

 Conversational Agent Chatbot Customer Service INTRODUCTION

Conversational Agents (CAs) have existed for many years since Eliza [ 11 ], the rst conversational agent (CA) was published in 1966. Since then, over the last few years they have drawn attention due to advancements in technology [ 5 ] [ 2 ] and the big amounts of data collected. Conversational Agents are becoming ubiquitous nowadays. They are used as personal assistants such as Google Assistant, Amazon Alexa, Apples Siri etc. Also they are used as customer service agents in call centers or in e-commerce for addressing customer needs and helping customers with online orders [ 3 ] [ 18 ] [ 4 ].

Conversational agents are used a lot in customer service. A recent report where 100 contact center leaders asked for their adoption to conversational agents speci ed that currently 46% of businesses are evaluating conversational agent's potential, while 32% have already adopted or are planning to adopt them soon [ 12 ]. Whilst CAs are exploding in popularity, it seems that they are not ready to serve customers as well as real agents [ 2 ]. There is not much research on the value they could bring to a business so that companies and customers could know why and how they should use them. The plan for this PhD project is to nd ways to measure the value of conversational agents and try to nd ways to increase this value. The value will be investigated from a customer perspective. Our main expected contribution will be: 1. Investigate the e ectiveness of Conversational Agents and identify challenges and limitations. 2. Investigate the customer's expectations and satisfaction of conversational agents and propose further improvements. 3. Create new optimised conversational agents for customer service. 4. Develop a theoretical framework for developing conversational agents for customer service.

The next sections of the paper will provide background work in CAs, and current and future work. 2

 BACKGROUND

There is a lot of research around Conversational Agents. Researchers are working in the back-end technologies and implementing state of the art deep learning and reinforcement learning models [ 15 ] [ 10 ] [ 16 ] to learn the CA to choose appropriate responses. To advance the technologies used for developing CAs, Amazon created the 'Alexa Prize competition' [ 14 ] in which di erent university's teams are developing open domain conversational agents.

Many publications exist for theoretical frameworks for conversational agents [ 13 ] [ 17 ] [ 1 ]. There is a lot of research on how to design a conversational agent in order to improve customer experience [ 19 ] [ 9 ] [ 8 ]. Also Gnewuch et al. investigated if fast or dynamic responses a ect the customer experience and also how the typing indicators a ect the customer experience [ 6 ] [ 7 ]. Hu et al. developed a tone aware conversational agent and investigated how this a ects the customer satisfaction [ 8 ].

Bringing all these together it is able to create CAs which can address customer needs. Almost half of the businesses are already investigating the potential of CAs. But what really are the bene ts of these Agents and why should companies invest in these? What is the real value that CAs can bring to a business and how can we measure this value? This is a gap which inspired us for doing this research. 3

 RESEARCH DESIGN

For measuring the value in this research, the CAs e ectiveness, challenges, limitations will be investigated by running experiments. Also customer's satisfaction will be measured when using these systems. We plan on simulating search tasks and investigate the previous aspects. Then the next step will be to use the ndings and embed them into new agents and run experiments with the optimised agents. We plan on comparing the new agents with the existed ones and measure the customer satisfaction in both scenarios. 3.1

Current Work

As a starting point we need to investigate and understand the e ectiveness of current CAs in E-commerce retailers in order to check if there is added value when using these systems. Also we want to identify the challenges and limitations that customers face when using these systems.

The Research Questions we are trying to address are: { RQ1: How e ective are the current conversational agents in E-commerce retailers? { RQ2: How satis ed are customers when using these systems?

In order to address this research questions we will try to answer the following secondary research questions: { Are existing conversational agents capable of addressing customer's information needs? What about complex information needs? { Is there statistically signi cant di erence in terms of time and usability when using a conversational agent instead of a website? { What is the level of satisfaction of customers when using this systems? { What are the biggest challenges and limitations that customers are facing when using conversational agents?

For answering these research questions, an experiment will be conducted in which participants will be asked to implement search tasks in E-commerce retail shops using existing CAs and websites and then they will be asked about their experience by completing surveys and discussion. 3.2

 Future Work

In the next years and after having the answers for the current research question, we plan on creating di erent CAs which will address some of the challenges and limitations and we will measure the added value when using these new CAs. In terms of customer satisfaction, we plan on embedding the results of current work in CAs in new optimised CAs and will investigate the customer satisfaction when using these new CAs. We plan on testing a variety of characteristics that can a ect the customer satisfaction and if time permits we plan on moving one step further in this research which is personalization. We should then create CAs that can perform in a personalized way with customers and investigate what is customer experience and the added value when using CAs which can function in a personal way.

 ACKNOWLEDGEMENTS

I would like to thank my supervisors Dr. Leif Azzopardi and Prof. Alan Wilson for their support and guidance to this project.

 1. Azzopardi , L. , Halvey , M. , Dalton , J.: Conversational search process . In :The Second International Workshop Conceptualizing Agent- Human Inter-actions During the Conversational Search Process ( 2018 ) 2. Bae Brandtzaeg , P. , Flstad , A. : Chatbots: changing user needs and motivations , Interactions Volume25 , 3843 ( 2018 ) 3. Chung , M. , Ko , E. , Joung , H. , Kim , S.J.: Chatbot e-service and customer satisfaction regarding luxury brands . Journal of Business Research ( 2018 ). 4. Cui , L. , Huang , S. , Wei , F. , Tan , C. , Duan , C. , Zhou , M.: SuperAgent: A Customer Service Chatbot for E-commerce Websites pp . 97102 . 5. Dale , R.: Industry Watch The return of the chatbots 22(5) , 811817 ( 2016 ) 6. Gnewuch , U. , Adam , M.T.P. , Maedche , A. : The Chatbot is typing . . . The Role of Typing Indicators in Human-Chatbot Interaction pp . 05 ( 2018 ) 7. Gnewuch , U. , Morana , S. , Adam , M. , Maedche , A. : Faster is Not Always Better: Understanding the E ect of Dynamic Response Delays in Human-Chatbot Interaction ( 2018 ) 8. Hu , T. , Xu , A. , Liu , Z. , You , Q. , Guo , Y. , Sinha , V. , Luo , J. , Akkiraju , R.: Touch Your Heart: A Tone-aware Chatbot for Customer Care on Social Media ( 2018 ) 9. Kiseleva , J.: Predicting User Satisfaction with Intelligent Assistants ( 2016 ) 10. Li , J. , Monroe , W. , Ritter , A. , Galley , M. , Gao , J. , Jurafsky , D. : Deep Reinforcement Learning for Dialogue Generation . Tech. rep. 11. J. Weizenbaum. ELIZA a computer program for the study of natural language communication between man and machine . Communications ofthe ACM , 9 : 3645 , ( 1966 ) 12. MYcustomer: Contact Centre Technology Barometer 2019 . A study of current and future technology adoption ( 2019 ), https://www.mycustomer.com/resources/contact -centre-technology-barometer2019 13. Radlinski , F. , Craswell , N.: A Theoretical Framework for Conversational Search . ( 2017 ) 14. Ram , A. , Prasad , R. , Khatri , C. , Venkatesh , A. , Gabriel , R., Liu, Q. , Nunn , J. , Hedayatnia , B., Cheng, M., Nagar , A. , King , E. , Bland , K. , Wartick , A. , Pan , Y. , Song , H. , Jayadevan , S. , Hwang , G. , Pettigrue , A. , Prize , A. : Conversational AI : The Science Behind the Alexa Prize . Tech. rep. ( 2017 ) 15. Sordoni , A. , Galley , M. , Auli , Brockett, C. , Ji , Y., Mitchell, M. , Nie , J.Y. , Gao , J. , Dolan , B. : A Neural Network Approach to Context-Sensitive Generation of Conversational Responses . Tech. rep. 16. Sutskever Google , I. , Vinyals Google , O. , Le Google , Q.V. : Sequence to Sequence Learning with Neural Networks . Tech. rep. 17. Trippas , J. R. , Spina , D. , Cavedon , L. , Joho , H. , Sanderson , M. : Informing the Design of Spoken Conversational Search ( 2018 ) 18. Xu , A. , Liu , Z. , Guo , Y. , Sinha , V. , Akkiraju , R.: A New Chatbot for Customer Service on Social Media ( 2017 ) 19. Zhang , L. , Webb , H. C.: E ects of emotional text on Online Customer Service Chat . Tech. rep.( 2010 )