We study human-chatbot collaborative conversation systems that enable humans to leverage AI chatbot outputs during an online conversation with others. We evaluate response quality in two collaborative systems and compare them with human-only and chatbot-only settings. Both collaborative systems present AI chatbot results as suggestions but encourage the synthesis of human and chatbot responses to different extents. We also examine the influence of chatbot choices, including both retrieval-based and generation-based methods, and the number of suggestions on collaborative systems. Experimental results show that our collaborative systems can significantly improve the efficiency to formulate a response and improve its quality compared with a human-only system while sacrificing the fluency and humanness of the messages. Compared with a chatbot, collaborative systems can provide answers that are more fluent, human-like, and informative. We also found that the retrieval-based chatbots perform better than the generation-based one from all aspects. The optimal number of chatbot suggestions is one, and showing more suggestions has reduced user efficiency. / Master of Science / Artificial Intelligence (AI) systems have become remarkably interactive and accurate with them becoming an integral part of our life. The increasing use of personal assistants like Siri and the application of AI in important real-world tasks such as medical imaging and diagnosis show that AI can perform as good as trained human experts. Organizations today are expanding at a rapid rate and need to service millions of customers concurrently to remain competitive in the market. With the recent success of AI chatbots, the collaboration of Human and AI to augment customer service management is one of the most sought out solutions to this requirement. A service flow where virtual agents and people work together can be a boon to the industry by making the human agents smarter with a bot "whispering" in their ears. We present the design of various collaborative systems we have developed and discuss the improvements in response efficiency and quality due to them in multiple online user experiments. The results of this study can be used to improve conversational chat systems that assist human agents to improve their response time and quality and identify features of the AI agent that are most beneficial for improving the conversation.
| Identifer | oai:union.ndltd.org:VTETD/oai:vtechworks.lib.vt.edu:10919/106683 |
| Date | 27 May 2020 |
| Creators | Ahuja, Naman |
| Contributors | Computer Science, Jiang, Jiepu, Ribbens, Calvin J., Karpatne, Anuj |
| Publisher | Virginia Tech |
| Source Sets | Virginia Tech Theses and Dissertation |
| Detected Language | English |
| Type | Thesis |
| Format | ETD, application/pdf, application/vnd.openxmlformats-officedocument.wordprocessingml.document, application/pdf |
| Rights | In Copyright, http://rightsstatements.org/vocab/InC/1.0/ |
Page generated in 0.0025 seconds