Dans une interaction humain-agent, l’engagement de l’utilisateur est un élément essentiel pour atteindre l’objectif de l’interaction. Dans cette thèse, nous étudions comment l’engagement de l’utilisateur pourrait être favorisé par le comportement de l’agent. Nous nous concentrons sur les stratégies de comportement verbal de l’agent qui concernent respectivement la forme, le timing et le contenu de ses énoncés. Nous présentons des études empiriques qui concernent certains aspects du comportement de politesse de l’agent, du comportement d’interruption de l’agent, et les sujets de conversation que l’agent adresse lors de l’interaction. Basé sur les résultats de la dernière étude, nous proposons un Gestionnaire de Sujets axé sur l’engagement (modèle computationnel) qui personnalise les sujets d’une interaction dans des conversations où l’agent donne des informations à un utilisateur humain. Le Modèle de Sélection des Sujets du Gestionnaire de Sujets décide sur quoi l’agent devrait parler et quand. Pour cela, il prend en compte la perception par l’agent de l’utilisateur, qui est dynamiquement mis à jour, ainsi que l’état mental et les préférences de l’agent. Le Modèle de Transition de Sujets du Gestionnaire de Sujet, basé sur une étude empirique, calcule comment l’agent doit présenter les sujets dans l’interaction en cours sans perdre la cohérence de l’interaction. Nous avons implémenté et évalué le Gestionnaire de Sujets dans un agent virtuel conversationnel qui joue le rôle d’un visiteur dans un musée. / In human-agent interaction the engagement of the user is an essential aspect to complete the goal of the interaction. In this thesis we study how the user’s engagement could be favoured by the agent’s behaviour. We thereby focus on the agent’s verbal behaviour considering strategies that regard respectively the form, timing, and content of utterances : We present empirical studies that regard (aspects of) the agent’s politeness behaviour, interruption behaviour, and the topics that the agent addresses in the interaction. Based on the outcomes of the latter study we propose an engagement-driven Topic Manager (computational model) that personalises the topics of an interaction in human-agent information-giving chat. The Topic Selection component of the Topic Manager decides what the agent should talk about and when. For this it takes into account the agent’s dynamically updated perception of the user as well as the agent’s own mental state. The Topic Transition component of the Topic Manager, based upon an empirical study, computes how the agent should introduce the topics in the ongoing interaction without loosing the coherence of the interaction. We implemented and evaluated the Topic Manager in a conversational virtual agent that plays the role of a visitor in amuseum.

<p>Robots have become increasingly common in various sectors, such as manufacturing, healthcare, and service industries. With the growing demand for automation and the expectation for interactive and assistive capabilities, robots must learn to adapt to unpredictable environments like humans can. This necessitates the development of learning methods that can effectively enable robots to collaborate with humans, learn from them, and provide guidance. Human experts commonly teach their collaborators to perform tasks via a few demonstrations, often followed by episodes of coaching that refine the trainee’s performance during practice. Adopting a similar approach that facilitates interactions to teaching robots is highly intuitive and enables task experts to teach the robots directly. Learning from Demonstration (LfD) is a popular method for robots to learn tasks by observing human demonstrations. However, for contact-rich tasks such as cleaning, cutting, or writing, LfD alone is insufficient to achieve a good performance. Further, LfD methods are developed to achieve observed goals while ignoring actions to maximize efficiency. By contrast, we recognize that leveraging human social learning strategies of practice and coaching in conjunction enables learning tasks with improved performance and efficacy. To address the deficiencies of learning from demonstration, we propose a Coaching by Demonstration (CbD) framework that integrates LfD-based practice with sparse coaching interactions from a human expert.</p> <p><br></p> <p>The LfD-based practice in CbD was implemented as an end-to-end off-policy reinforcement learning (RL) agent with the action space and rewards inferred from the demonstration. By modeling the reward as a similarity network trained on expert demonstrations, we eliminate the need for designing task-specific engineered rewards. Representation learning was leveraged to create a novel state feature that captures interaction markers necessary for performing contact-rich skills. This LfD-based practice was combined with coaching, where the human expert can improve or correct the objectives through a series of interactions. The dynamics of interaction in coaching are formalized using a partially observable Markov decision process. The robot aims to learn the true objectives by observing the corrective feedback from the human expert. We provide an approximate solution by reducing this to a policy parameter update using KL divergence between the RL policy and a Gaussian approximation based on coaching. The proposed framework was evaluated on a dataset of 10 contact-rich tasks from the assembly (peg-insertion), service (cleaning, writing, peeling), and medical domains (cricothyroidotomy, sonography). Compared to baselines of behavioral cloning and reinforcement learning algorithms, CbD demonstrates improved performance and efficiency.</p> <p><br></p> <p>During the learning process, the demonstrations and coaching feedback imbue the robot with expert knowledge of the task. To leverage this expertise, we develop a reverse coaching model where the robot can leverage knowledge from demonstrations and coaching corrections to provide guided feedback to human trainees to improve their performance. Providing feedback adapted to individual trainees' "style" is vital to coaching. To this end, we have proposed representing style as objectives in the task null space. Unsupervised clustering of the null-space trajectories using Gaussian mixture models allows the robot to learn different styles of executing the same skill. Given the coaching corrections and style clusters database, a style-conditioned RL agent was developed to provide feedback to human trainees by coaching their execution using virtual fixtures. The reverse coaching model was evaluated on two tasks, a simulated incision and obstacle avoidance through a haptic teleoperation interface. The model improves human trainees’ accuracy and completion time compared to a baseline without corrective feedback. Thus, by taking advantage of different human-social learning strategies, human-robot collaboration can be realized in human-centric environments. </p> <p><br></p>

The rapid growth of textual data in the digital age presents unique challenges in domain-specific text classification, particularly the scarcity of labeled data for many applications, due to expensive cost of manual labeling work. In this thesis, we explore the applicability of prompt-learning method, which is well-known for being suitable in few-shot scenarios and much less data-consuming, as an emerging alternative to traditional fine-tuning methods, for domain-specific text classification in the context of customer-agent interactions in the retail sector. Specifically, we implemented the entire prompt-learning pipeline for the classification task, and, our investigation encompasses various strategies of prompt-learning, including fixed-prompt language model tuning strategy and tuning-free prompting strategy, along with an examination of language model selection, few-shot sampling strategy, prompt template design, and verbalizer design. In this manner, we assessed the overall performance of the prompt-learning method in the classification task. Through a systematic evaluation, we demonstrate that with the fixed-prompt language model tuning strategy, based on relatively smaller language models (e.g. T5-base with around 220M parameters), prompt-learning can achieve competitive performance (close to 75% accuracy) even with limited labeled data (up to merely 15% of full data). And besides, with the tuning-free prompting strategy, based on a regular-size language model (e.g. FLAN-T5-large with around 770M parameters), the performance can be up to around 30% accuracy with detailed prompt templates and zero-shot setting (no extra training data involved). These results can offer valuable insights for researchers and practitioners working with domain-specific textual data, prompt-learning and few-shot / zero-shot learning. The findings of this thesis highlight the potential of prompt-learning as a practical solution for classification problems across diverse domains and set the stage for future research in this area.