Embodied Conversational Agents (ECAs) are intelligent software entities with an embodiment used to communicate with users, using natural language. Their purpose is to exhibit the same properties as humans in face-to-face conversation, including the ability to produce and respond to verbal and nonverbal communication. Researchers in the field of ECAs try to create agents that can be more natural, believable and easy to use. Designing an ECA requires understanding that manner, personality, emotion, and appearance are very important issues to be considered. In this thesis, we are interested in increasing believability of ECAs by placing personality at the heart of the human-agent verbal interaction. We propose a model relating personality facets and hidden communication goals that can influence ECA behaviors. Moreover, we apply our model in agents that interact in a puzzle game application. We develop five distinct personality oriented agents using an expressive communication language and a plan-based BDI approach for modeling and managing dialogue according to our proposed model. In summary, we present and test an innovative approach to model mental aspects of ECAs trying to increase their believability and to enhance human-agent affective communication. With this research, we hope to improve the understanding on how ECAs with expressive and affective characteristics can establish and maintain long-term human-agent relationships.

The aim of this work is to simplify access to information for citizens of the city of Brno and at the same time to innovate the way of communication between the citizen and his city. The problem is solved by creating a conversational agent - chatbot Kroko. Using artificial intelligence and a Czech language analyzer, the agent is able to understand and respond to a certain set of textual, natural language queries. The agent is available on the Messenger platform and has a knowledge base that includes data provided by the city council. After conducting an extensive user testing on a total of 76 citizens of the city, it turned out that up to 97\% of respondents like the idea of a city-oriented chatbot and can imagine using it regularly. The main finding of this work is that the general public can easily adopt and effectively use a chatbot. The results of this work motivate further development of practical applications of conversational agents.

As Digital Personal Assistants get increasingly present in our lives, repositioning Conversational Interfaces within Interaction Design could be beneficial. More contributions seem possible beyond the commercial vision of Conversational Agents as digital assistants. In this thesis, Design fiction is adopted as an approach to explore a future for these technologies, focusing on the possible social and ritual practices that might arise when Conversational Agents and Artificial Intelligence are employed in contexts such as mortality and grief. As a secondary but related concern, it is argued that designers need to come to find ways to work with Artificial Intelligence and Machine Learning, uncovering the “AI blackbox”, and understanding its basic functioning, therefore, Machine learning is explored as a design material. This research through design project presents a scenario where the data we leave behind us are used after we die to build conversational models that become digital altars, shaping the way we deal with grief and death. This is presented through a semi-functional prototype and a diegetic prototype in the form of a short video.

Les agents virtuels conversationnels ayant un comportement social reposent souvent sur au moins deux disciplines différentes : l’informatique et la psychologie. Dans la plupart des cas, les théories psychologiques sont converties en un modèle informatique afin de permettre aux agents d’adopter des comportements crédibles. Nos travaux de thèse se positionnent au croisement de ces deux champs disciplinaires. Notre objectif est de renforcer la crédibilité des agents conversationnels. Nous nous intéressons aux agents conversationnels orientés tâche, qui sont utilisés dans un contexte professionnel pour produire des réponses à partir d’une base de connaissances métier. Nous proposons un modèle affectif pour ces agents qui s’inspire des mécanismes affectifs chez l’humain. L’approche que nous avons choisie de mettre en œuvre dans notre modèle s’appuie sur la théorie des Tendances à l’Action en psychologie. Nous avons proposé un modèle des émotions en utilisant un formalisme inspiré de la logique BDI pour représenter les croyances et les buts de l’agent. Ce modèle a été implémenté dans une architecture d’agent conversationnel développée au sein de l’entreprise DAVI. Afin de confirmer la pertinence de notre approche, nous avons réalisé plusieurs études expérimentales. La première porte sur l’évaluation d’expressions verbales de la tendance à l’action. La deuxième porte sur l’impact des différentes stratégies de régulation possibles sur la perception de l’agent par l’utilisateur. Enfin, la troisième étude porte sur l’évaluation des agents affectifs en interaction avec des participants. Nous montrons que le processus de régulation que nous avons implémenté permet d’augmenter la crédibilité et le professionnalisme perçu des agents, et plus généralement qu’ils améliorent l’interaction. Nos résultats mettent ainsi en avant la nécessité de prendre en considération les deux mécanismes émotionnels complémentaires : la génération et la régulation des réponses émotionnelles. Ils ouvrent des perspectives sur les différentes manières de gérer les émotions et leur impact sur la perception de l’agent. / Conversational virtual agents with social behavior are often based on at least two different disciplines : computer science and psychology. In most cases, psychological findings are converted into computational mechanisms in order to make agents look and behave in a believable manner. In this work, we aim at increasing conversational agents’ belivielibity and making human-agent interaction more natural by modelling emotions. More precisely, we are interested in task-oriented conversational agents, which are used as a custumer-relationship channel to respond to users request. We propose an affective model of emotional responses’ generation and control during a task-oriented interaction. Our proposed model is based, on one hand, on the theory of Action Tendencies (AT) in psychology to generate emotional responses during the interaction. On the other hand, the emotional control mechanism is inspired from social emotion regulation in empirical psychology. Both mechanisms use agent’s goals, beliefs and ideals. This model has been implemented in an agent architecture endowed with a natural language processing engine developed by the company DAVI. In order to confirm the relevance of our approach, we realized several experimental studies. The first was about validating verbal expressions of action tendency in a human-agent dialogue. In the second, we studied the impact of different emotional regulation strategies on the agent perception by the user. This study allowed us to design a social regulation algorithm based on theoretical and empirical findings. Finally, the third study focuses on the evaluation of emotional agents in real-time interactions. Our results show that the regulation process contributes in increasing the credibility and perceived competence of agents as well as in improving the interaction. Our results highlight the need to take into consideration of the two complementary emotional mechanisms : the generation and regulation of emotional responses. They open perspectives on different ways of managing emotions and their impact on the perception of the agent.

This study focuses on evaluating and enhancing the user experience of Emely, a conversational agent aimed at improving language skills for second language learners, particularly those who want to increase their chances of securing employment in Sweden. Usability testing was conducted in two test rounds, with the first round providing design implications for the user interface in the second round. However, assessing the effectiveness of the interface improvements was challenging due to low Swedish proficiency among the test groups consisting of potential users of Emely. Although the study did not result in design implications for the user interface, important findings highlight the need to adapt conversational agents, like Emely, for users with low literacy levels and illiteracy, emphasizing the importance of inclusive design for effective language learning support.

Assessing a conversational agent’s understanding capabilities is critical, as poor user interactions could seal the agent’s fate at the very beginning of its lifecycle with users abandoning the system. In this thesis we explore the use of paraphrases as a testing tool for conversational agents. Paraphrases, which are different ways of expressing the same intent, are generated based on known working input by performing lexical substitutions and by introducing multiple spelling divergences. As the expected outcome for this newly generated data is known, we can use it to assess the agent’s robustness to language variation and detect potential understanding weaknesses. As demonstrated by a case study, we obtain encouraging results as it appears that this approach can help anticipate potential understanding shortcomings, and that these shortcomings can be addressed by the generated paraphrases. / Att bedöma en konversationsagents språkförståelse är kritiskt, eftersom dåliga användarinteraktioner kan avgöra om agenten blir en framgång eller ett misslyckande redan i början av livscykeln. I denna rapport undersöker vi användningen av parafraser som ett testverktyg för dessa konversationsagenter. Parafraser, vilka är olika sätt att uttrycka samma avsikt, skapas baserat på känd indata genom att utföra lexiska substitutioner och genom att introducera flera stavningsavvikelser. Eftersom det förväntade resultatet för denna indata är känd kan vi använda resultaten för att bedöma agentens robusthet mot språkvariation och upptäcka potentiella förståelssvagheter. Som framgår av en fallstudie får vi uppmuntrande resultat, eftersom detta tillvägagångssätt verkar kunna bidra till att förutse eventuella brister i förståelsen, och dessa brister kan hanteras av de genererade parafraserna.

As research in Artificial Intelligence (AI) advances, it is crucial to focus on having seamless communication between humans and machines in order to effectively accomplish tasks. Smooth human-machine communication requires the machine to be sensible and human-like while interacting with humans, while simultaneously being capable of extracting the maximum information it needs to accomplish the desired task. Since a lot of the tasks required to be solved by machines today involve the understanding of images, training machines to have human-like and effective image-grounded conversations with humans is one important step towards achieving this goal. Although we now have agents that can answer questions asked for images, they are prone to failure from confusing input, and cannot ask clarification questions, in turn, to extract the desired information from humans. Hence, as a first step, we direct our efforts towards making Visual Question Answering agents human-like by making them resilient to confusing inputs that otherwise do not confuse humans. Not only is it crucial for a machine to answer questions reasonably, it should also know how to ask questions sequentially to extract the desired information it needs from a human. Hence, we introduce a novel game called the Visual 20 Questions Game, where a machine tries to figure out a secret image a human has picked by having a natural language conversation with the human. Using deep learning techniques like recurrent neural networks and sequence-to-sequence learning, we demonstrate scalable and reasonable performances on both the tasks. / Master of Science / Research in Artificial Intelligence has reached to a point where computers can answer natural freeform questions asked to arbitrary images in a somewhat reasonable manner. These machines are called Visual Question Answering agents. However, they are prone to failure from even a slightly confusing input. For example, for an obviously irrelevant question asked to an image, they would answer something non-sensical instead of recognizing that the question is irrelevant. Furthermore, they also cannot ask questions in turn to humans for clarification or for more information. These shortcomings not only harm their efficacy, but also harm their perceived trust from human users. In order to remedy these problems, we first direct our efforts towards making Visual Question Answering agents capable of identifying an irrelevant question for an image. Next, we also try to train machines to be able to ask questions to extract more information from humans to make an informed decision. We do this by introducing a novel game called the Visual 20 Questions game, where a machine tries to figure out a secret image a human has picked by having a natural language conversation with the human. Deep learning techniques such as sequence-to-sequence learning using recurrent neural networks make it possible for machines to learn how to converse based on a series of conversational exchanges made between two humans. Techniques like reinforcement learning make it possible for machines to better themselves based on rewards it gets for accomplishing a task in a certain way. Using such algorithms, we demonstrate promise towards scalable and reasonable performances on both the tasks.

This thesis presents a framework for computer assisted language learning using a virtual language teacher. It is an attempt at creating, not only a new type of language learning software, but also a server-based application that collects large amounts of speech material for future research purposes.The motivation for the framework is to create a research platform for computer assisted language learning, and computer assisted pronunciation training.Within the thesis, different feedback strategies and pronunciation error detectors are exploredThis is a broad, interdisciplinary approach, combining research from a number of scientific disciplines, such as speech-technology, game studies, cognitive science, phonetics, phonology, and second-language acquisition and teaching methodologies.The thesis discusses the paradigm both from a top-down point of view, where a number of functionally separate but interacting units are presented as part of a proposed architecture, and bottom-up by demonstrating and testing an implementation of the framework. / QC 20110511

Research endeavors on spoken dialogue systems in the 1990s and 2000s have led to the deployment of commercial spoken dialogue systems (SDS) in microdomains such as customer service automation, reservation/booking and question answering systems. Recent research in SDS has been focused on the development of applications in different domains (e.g. virtual counseling, personal coaches, social companions) which requires more sophistication than the previous generation of commercial SDS. The focus of this research project is the delivery of behavior change interventions based on the brief intervention counseling style via spoken dialogue systems. Brief interventions (BI) are evidence-based, short, well structured, one-on-one counseling sessions. Many challenges are involved in delivering BIs to people in need, such as finding the time to administer them in busy doctors' offices, obtaining the extra training that helps staff become comfortable providing these interventions, and managing the cost of delivering the interventions. Fortunately, recent developments in spoken dialogue systems make the development of systems that can deliver brief interventions possible. The overall objective of this research is to develop a data-driven, adaptable dialogue system for brief interventions for problematic drinking behavior, based on reinforcement learning methods. The implications of this research project includes, but are not limited to, assessing the feasibility of delivering structured brief health interventions with a data-driven spoken dialogue system. Furthermore, while the experimental system focuses on harmful alcohol drinking as a target behavior in this project, the produced knowledge and experience may also lead to implementation of similarly structured health interventions and assessments other than the alcohol domain (e.g. obesity, drug use, lack of exercise), using statistical machine learning approaches. In addition to designing a dialog system, the semantic and emotional meanings of user utterances have high impact on interaction. To perform domain specific reasoning and recognize concepts in user utterances, a named-entity recognizer and an ontology are designed and evaluated. To understand affective information conveyed through text, lexicons and sentiment analysis module are developed and tested.

This research paper presents a study exploring if using the Conversational Analysis (CA) method in design research could result in designing more socioculturally contextual conversational agents. The research specifically focused on understanding the 1) effect on the design outcome and 2) the role in the design process. This was studied through practice-based design research, participant evaluation of the design outcome, and expert interviews on the design method. The findings were analysed both qualitatively and quantitatively and showed, that socioculturally contextual design could potentially be a data-rich field of study with connections to design concepts such as inclusive design, affective design, design ethics, increased user experience, and further studies are therefore recommended. Furthermore, the study provided an understanding of the contexts in which the CA method may be useful in design, how it can potentially impact the design, and how to apply it to the design process and showed a positive effect on the design outcome in terms of socioculturally contextual design.