This thesis examines recommendation dialogue, in the context of dialogue strategy design for conversational recommender systems. The purpose of a recommender system is to produce personalized recommendations of potentially useful items from a large space of possible options. In a conversational recommender system, this task is approached by utilizing natural language recommendation dialogue for detecting user preferences, as well as for providing recommendations. The fundamental idea of a conversational recommender system is that it relies on dialogue sessions to detect, continuously update, and utilize the user's preferences in order to predict potential interest in domain items modeled in a system. Designing the dialogue strategy management is thus one of the most important tasks for such systems. Based on empirical studies as well as design and implementation of conversational recommender systems, a behavior-based dialogue model called bcorn is presented. bcorn is based on three constructs, which are presented in the thesis. It utilizes a user preference modeling framework (preflets) that supports and utilizes natural language dialogue, and allows for descriptive, comparative, and superlative preference statements, in various situations. Another component of bcorn is its message-passing formalism, pcql, which is a notation used when describing preferential and factual statements and requests. bcorn is designed to be a generic recommendation dialogue strategy with conventional, information-providing, and recommendation capabilities, that each describes a natural chunk of a recommender agent's dialogue strategy, modeled in dialogue behavior diagrams that are run in parallel to give rise to coherent, flexible, and effective dialogue in conversational recommender systems. Three empirical studies have been carried out in order to explore the problem space of recommendation dialogue, and to verify the solutions put forward in this work. Study I is a corpus study in the domain of movie recommendations. The result of the study is a characterization of recommendation dialogue, and forms a base for a first prototype implementation of a human-computer recommendation dialogue control strategy. Study II is an end-user evaluation of the acorn system that implements the dialogue control strategy and results in a verification of the effectiveness and usability of the dialogue strategy. There are also implications that influence the refinement of the model that are used in the bcorn dialogue strategy model. Study III is an overhearer evaluation of a functional conversational recommender system called CoreSong, which implements the bcorn model. The result of the study is indicative of the soundness of the behavior-based approach to conversational recommender system design, as well as the informativeness, naturalness, and coherence of the individual bcorn dialogue behaviors. / I denna avhandling undersöks rekommendationsdialog med avseende på utformningen av dialogstrategier f¨or konverserande rekommendationssystem. Syftet med ett rekommendationssystem är att generera personaliserade rekommendationer utifrån potentiellt användbara domänobjekt i stora informationsrymder. I ett konverserande rekommendationssystem angrips detta problem genom att utnyttja naturligt språkk och dialog för att modellera användarpreferenser, liksom för att ge rekommendationer. Grundidén med konverserande rekommendationssystem är att utnyttja dialogsessioner för att upptäcka, uppdatera och utnyttja en användares preferenser för att förutsäga användarens intresse för domänobjekten som modelleras i ett system. Utformningen av dialogstrategihantering är därför en av de viktigaste uppgifterna för sådana system. Baserat på empiriska studier, liksom på utformning och implementering av konverserande rekommendationssystem, presenteras en beteendebaserad dialogmodell som kallas bcorn. bcorns bas utgörs av tre konstruktioner, vilka alla presenteras i denna avhandling. bcorn utnyttjar ett preferensmodelleringsramverk (preflets) som stöder och anv¨ander sig av naturligt språk i dialog och tillåter deskriptiva, komparativa och superlativa preferensuttryck i olika situationer. Den andra komponenten i bcorn är dess interna meddelande-formalism pcql, som är en notation som kan beskriva preferens- och faktiska påståenden och frågor. bcorn är utformat som en generell rekommendationshanteringsstrategi med konventionella, informationsgivande och rekommenderande förmågor, som var och en beskriver naturliga delar av en rekommendationsagents dialogstrategi. Dessa delar modelleras i dialogbeteendediagram som exekveras parallellt för att ge upphov till koherent, flexibel och effektiv dialog i konverserande rekommendationssystem. Tre empiriska studier har utförts för att utforska problemkomplexet som utgör rekommendationsdialog och för att verifiera de lösningar som tagits fram inom ramen för detta arbete. Studie I är en korpusstudie i filmrekommendationsdomänen. Studien resulterar i en karakteristik av rekommendationsdialog, och utgör basen för en första prototyp av dialoghanteringsstrategi för rekommendationsdialog mellan människa och dator. Studie II är en slutanvändarutvärdering av systemet acorn som implementerar denna dialoghanteringsstrategi och resulterar i en verifiering av effektivitet och användbarhet av strategin. Studien resulterar också i implikationer som påverkar utformningen av den modell som används i bcorn. Studie III är en medhörningsutvärdering av det funktionella konverserande rekommendationssystemet CoreSong, som implementerar bcorn-modellen. Resultatet av studien indikerar att det beteendebaserade angreppssättet är funktionellt och att de olika dialogbeteendena i bcorn ger upphov till h¨og informationskvalitet, naturlighet och koherens i rekommendationsdialog.

This thesis focuses on the problem of scalable optimization of dialogue behaviour in speech-based conversational systems using reinforcement learning. Most previous investigations in dialogue strategy learning have proposed flat reinforcement learning methods, which are more suitable for small-scale spoken dialogue systems. This research formulates the problem in terms of Semi-Markov Decision Processes (SMDPs), and proposes two hierarchical reinforcement learning methods to optimize sub-dialogues rather than full dialogues. The first method uses a hierarchy of SMDPs, where every SMDP ignores irrelevant state variables and actions in order to optimize a sub-dialogue. The second method extends the first one by constraining every SMDP in the hierarchy with prior expert knowledge. The latter method proposes a learning algorithm called 'HAM+HSMQ-Learning', which combines two existing algorithms in the literature of hierarchical reinforcement learning. Whilst the first method generates fully-learnt behaviour, the second one generates semi-learnt behaviour. In addition, this research proposes a heuristic dialogue simulation environment for automatic dialogue strategy learning. Experiments were performed on simulated and real environments based on a travel planning spoken dialogue system. Experimental results provided evidence to support the following claims: First, both methods scale well at the cost of near-optimal solutions, resulting in slightly longer dialogues than the optimal solutions. Second, dialogue strategies learnt with coherent user behaviour and conservative recognition error rates can outperform a reasonable hand-coded strategy. Third, semi-learnt dialogue behaviours are a better alternative (because of their higher overall performance) than hand-coded or fully-learnt dialogue behaviours. Last, hierarchical reinforcement learning dialogue agents are feasible and promising for the (semi) automatic design of adaptive behaviours in larger-scale spoken dialogue systems. This research makes the following contributions to spoken dialogue systems which learn their dialogue behaviour. First, the Semi-Markov Decision Process (SMDP) model was proposed to learn spoken dialogue strategies in a scalable way. Second, the concept of 'partially specified dialogue strategies' was proposed for integrating simultaneously hand-coded and learnt spoken dialogue behaviours into a single learning framework. Third, an evaluation with real users of hierarchical reinforcement learning dialogue agents was essential to validate their effectiveness in a realistic environment.

Les développements issus de la recherche en gestion du dialogue homme-machine portent essentiellement sur le dialogue utilitaire et délaissent le dialogue à caractère ludique ou récréatif. Une description détaillée du contexte et la reconnaissance des buts ne suffisent pas à appréhender les enjeux de ce type de dialogue. La thèse vise à démontrer qu'un système s'appuyant sur une reconnaissance robuste et fine des actes de dialogue et des intentions associée à une prise en compte de l'altérité, de l'éthique et des émotions peut faire émerger une personnalité dialogique à même d'interagir de façon crédible avec un humain et de reproduire certaines performances dans le contexte d'un dialogue de séduction. L'objectif ne consiste pas à faire en sorte que le système puisse être confondu avec un humain comme cela est le cas dans les tests d'intelligence mais plutôt faire en sorte que le système puisse être dirigé par l'intérêt de la conversation. A cette fin, les recherches portent sur la définition de la relation d'altérité JE-TU appliquée au dialogue par l'intermédiaire de la théorie des jeux notamment à travers les concepts de définition de soi, d'éthique de la discussion et de modèles d'émotions. Plusieurs pistes sont explorées dans le but de réunir un corpus d'étude, notamment des prototypes de jeux collaboratifs. Au final, le modèle de personnage est développé sur la base d'un corpus de scripts de cinéma. Ce modèle repose sur une nouvelle taxonomie de phénomènes de dialogue incluant des actes perlocutoires et une approche différente de la connaissance permettant d'inclure l'éthique et le lien d'altérité en son sein. Les stratégies qui régissent le dialogue sont alors décrites de manière beaucoup plus précise. Une scène extraite d'un film servant de cadre applicatif aux expérimentations permet de valider l'architecture du système de dialogue. / Most of the developments issued from the research in man computer dialogue management are mainly focused on functional dialogue and put aside entertainment dialogue. A sharper context description and goals recognition remains quite limited to fully grasp the stakes of type of dialogue. The thesis aims to demonstrate that a system based on a thin and robust recognition of dialogue acts and intentions linked to a consideration for the concept of relation of otherness, ethics and emotions could leads to the emergence of a dialogic personality. Such a system would be able to interact with a human being in a credible way and reproduce some of its achievements in the context of a seduction dialogue. Unlike the common approach developed for intelligence tests, the purpose is not to mimic a human, but to manage the dialogue based on the notion of interest. To achieve this goal, researches deal with the relation of otherness applied to dialogue within game theory and especially through the concepts of Self-definition, Discourse Ethics and Emotions Modeling. The corpora collection process follows many ways including collaborative games. At the end, the character model is developed using a movie script corpora on the basis of a new dialogue phenomena taxonomy including perlocutionary acts and a new approach of knowledge that incorporate ethics and the relation of otherness. It leads to a thinner description of dialogue strategies. A scene extracted from a movie aims to validate the final architecture as an experimental framework.

<p>У овој дисертацији се разматрају изабрани аспекти истраживачког проблема спецификације, дизајнирања и имплементације конверзационих робота као асистивних средстава у терапији деце са церебралном парализом. Доприноси ови тезе су следећи. (i) Предложена је архитектура конверзационог агента опште намене која омогућава флексибилно интегрисање модула различитих функционалности. (ii) Дефинисана је и имплементирана адаптивна бихевиористичка стратегија коју робот примењује у интеракцији са децом. (iii) Предложена дијалошка стратегија је спроведена и позитивно процењена у интеркацији између деце и робота у реалистичном терапеутском контексту. (iv) Коначно, предложен је приступ за аутоматско детектовање критичних промена у дијалогу, заснован на појму нормализоване дијалошке ентропије.</p> / <p>U ovoj disertaciji se razmatraju izabrani aspekti istraživačkog problema specifikacije, dizajniranja i implementacije konverzacionih robota kao asistivnih sredstava u terapiji dece sa cerebralnom paralizom. Doprinosi ovi teze su sledeći. (i) Predložena je arhitektura konverzacionog agenta opšte namene koja omogućava fleksibilno integrisanje modula različitih funkcionalnosti. (ii) Definisana je i implementirana adaptivna bihevioristička strategija koju robot primenjuje u interakciji sa decom. (iii) Predložena dijaloška strategija je sprovedena i pozitivno procenjena u interkaciji između dece i robota u realističnom terapeutskom kontekstu. (iv) Konačno, predložen je pristup za automatsko detektovanje kritičnih promena u dijalogu, zasnovan na pojmu normalizovane dijaloške entropije.</p> / <p>This doctoral dissertation considers selected aspects of the research problem of specification, design, and implementation of conversational robots as assistive tools in therapy for children with cerebral palsy. This dissertation has made the following contributions: (i) It proposes a general architecture for conversational agents that allows for flexible integration of software modules implementing different functionalities. (ii) It introduces and implements an adaptive behavioural strategy that is applied by the robot in interaction with children. (iii) The proposed dialogue strategy is applied and evaluated in interaction between children and the robot MARKO, in realistic therapeutic settings. (iv) Finally, the dissertation proposes an approach to automatic detection of critical changes in human-machine interaction, based on the notion of normalized interactional entropy.</p>