Recent advancements in the area of spoken language processing and the wide acceptance of portable devices, have attracted signicant interest in spoken dialogue systems.These conversational systems are man-machine interfaces which use natural language (speech) as the medium of interaction.In order to conduct dialogues, computers must have the ability to decide when and what information has to be exchanged with the users. The dialogue management module is responsible to make these decisions so that the intended task (such as ticket booking or appointment scheduling) can be achieved.Thus learning a good strategy for dialogue management is a critical task.In recent years reinforcement learning-based dialogue management optimization has evolved to be the state-of-the-art. A majority of the algorithms used for this purpose needs vast amounts of training data.However, data generation in the dialogue domain is an expensive and time consuming process. In order to cope with this and also to evaluatethe learnt dialogue strategies, user modelling in dialogue systems was introduced. These models simulate real users in order to generate synthetic data.Being computational models, they introduce some degree of modelling errors. In spite of this, system designers are forced to employ user models due to the data requirement of conventional reinforcement learning algorithms can learn optimal dialogue strategies from limited amount of training data when compared to the conventional algorithms. As a consequence of this, user models are no longer required for the purpose of optimization, yet they continue to provide a fast and easy means for quantifying the quality of dialogue strategies. Since existing methods for user modelling are relatively less realistic compared to real user behaviors, the focus is shifted towards user modelling by means of inverse reinforcement learning. Using experimental results, the proposed method's ability to learn a computational models with real user like qualities is showcased as part of this work.

The purpose of this thesis is to develop and evaluate user simulators for a spoken dialogue system. Created simulators are operating on dialogue act level. We implemented a bigram simulator as a baseline system. Based on the baseline simulator, we created another bigram simulator that is trained on dialogue acts without slot values. The third implemented simulator is similar to an implemen- tation of a dialogue manager. It tracks its dialogue state and learns a dialogue strategy based on the state using supervised learning. The user simulators are implemented in Python 2.7, in ALEX framework for dialogue system development. Simulators are developed for PTICS application which operates in the domain of public transport information. Simulators are trained and evaluated using real human-machine dialogues collected with PTICS application. 1

Les récents progrès dans le domaine du traitement du langage ont apporté un intérêt significatif à la mise en oeuvre de systèmes de dialogue parlé. Ces derniers sont des interfaces utilisant le langage naturel comme medium d'interaction entre le système et l'utilisateur. Le module de gestion de dialogue choisit le moment auquel l'information qu'il choisit doit être échangée avec l'utilisateur. Ces dernières années, l'optimisation de dialogue parlé en utilisant l'apprentissage par renforcement est devenue la référence. Cependant, une grande partie des algorithmes utilisés nécessite une importante quantité de données pour être efficace. Pour gérer ce problème, des simulations d'utilisateurs ont été introduites. Cependant, ces modèles introduisent des erreurs. Par un choix judicieux d'algorithmes, la quantité de données d'entraînement peut être réduite et ainsi la modélisation de l'utilisateur évitée. Ces travaux concernent une partie des contributions présentées. L'autre partie des travaux consiste à proposer une modélisation à partir de données réelles des utilisateurs au moyen de l'apprentissage par renforcement inverse / Recent advancements in the area of spoken language processing and the wide acceptance of portable devices, have attracted signicant interest in spoken dialogue systems.These conversational systems are man-machine interfaces which use natural language (speech) as the medium of interaction.In order to conduct dialogues, computers must have the ability to decide when and what information has to be exchanged with the users. The dialogue management module is responsible to make these decisions so that the intended task (such as ticket booking or appointment scheduling) can be achieved.Thus learning a good strategy for dialogue management is a critical task.In recent years reinforcement learning-based dialogue management optimization has evolved to be the state-of-the-art. A majority of the algorithms used for this purpose needs vast amounts of training data.However, data generation in the dialogue domain is an expensive and time consuming process. In order to cope with this and also to evaluatethe learnt dialogue strategies, user modelling in dialogue systems was introduced. These models simulate real users in order to generate synthetic data.Being computational models, they introduce some degree of modelling errors. In spite of this, system designers are forced to employ user models due to the data requirement of conventional reinforcement learning algorithms can learn optimal dialogue strategies from limited amount of training data when compared to the conventional algorithms. As a consequence of this, user models are no longer required for the purpose of optimization, yet they continue to provide a fast and easy means for quantifying the quality of dialogue strategies. Since existing methods for user modelling are relatively less realistic compared to real user behaviors, the focus is shifted towards user modelling by means of inverse reinforcement learning. Using experimental results, the proposed method's ability to learn a computational models with real user like qualities is showcased as part of this work.

El objetivo principal de la tesis que se presenta es el estudio y desarrollo de diferentes metodologías para la gestión del diálogo en sistemas de diálogo hablado. El principal reto planteado en la tesis reside en el desarrollo de metodologías puramente estadísticas para la gestión del diálogo, basadas en el aprendizaje de un modelo a partir de un corpus de diálogos etiquetados. En este campo, se presentan diferentes aproximaciones para realizar la gestión, la mejora del modelo estadístico y la evaluación del sistema del diálogo. Para la implementación práctica de estas metodologías, en el ámbito de una tarea específica, ha sido necesaria la adquisición y etiquetado de un corpus de diálogos. El hecho de disponer de un gran corpus de diálogos ha facilitado el aprendizaje y evaluación del modelo de gestión desarrollado. Así mismo, se ha implementado un sistema de diálogo completo, que permite evaluar el funcionamiento práctico de las metodologías de gestión en condiciones reales de uso. Para evaluar las técnicas de gestión del diálogo se proponen diferentes aproximaciones: la evaluación mediante usuarios reales; la evaluación con el corpus adquirido, en el cual se han definido unas particiones de entrenamiento y prueba; y la utilización de técnicas de simulación de usuarios. El simulador de usuario desarrollado permite modelizar de forma estadística el proceso completo del diálogo. En la aproximación que se presenta, tanto la obtención de la respuesta del sistema como la generación del turno de usuario se modelizan como un problema de clasificación, para el que se codifica como entrada un conjunto de variables que representan el estado actual del diálogo y como resultado de la clasificación se obtienen las probabilidades de seleccionar cada una de las respuestas (secuencia de actos de diálogo) definidas respectivamente para el usuario y el sistema. / Griol Barres, D. (2007). Desarrollo y evaluación de diferentes metodologías para la gestión automática del diálogo [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/1956