Trust between humans and artificial systems has recently received increased attention due to the widespread use of autonomous systems in our society. In this context trust plays a dual role. On the one hand it is necessary to build robots that are perceived as trustworthy by humans. On the other hand we need to give to those robots the ability to discriminate between reliable and unreliable informants. This thesis focused on the second problem, presenting an interdisciplinary investigation of trust, in particular a computational model based on neuroscientific and psychological assumptions. First of all, the use of Bayesian networks for modelling causal relationships was investigated. This approach follows the well known theory-theory framework of the Theory of Mind (ToM) and an established line of research based on the Bayesian description of mental processes. Next, the role of gaze in human-robot interaction has been investigated. The results of this research were used to design a head pose estimation system based on Convolutional Neural Networks. The system can be used in robotic platforms to facilitate joint attention tasks and enhance trust. Finally, everything was integrated into a structured cognitive architecture. The architecture is based on an actor-critic reinforcement learning framework and an intrinsic motivation feedback given by a Bayesian network. In order to evaluate the model, the architecture was embodied in the iCub humanoid robot and used to replicate a developmental experiment. The model provides a plausible description of children's reasoning that sheds some light on the underlying mechanism involved in trust-based learning. In the last part of the thesis the contribution of human-robot interaction research is discussed, with the aim of understanding the factors that influence the establishment of trust during joint tasks. Overall, this thesis provides a computational model of trust that takes into account the development of cognitive abilities in children, with a particular emphasis on the ToM and the underlying neural dynamics.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第18410号 / 情博第525号 / 新制||情||93(附属図書館) / 31268 / 京都大学大学院情報学研究科知能情報学専攻 / (主査)教授 奥乃 博, 教授 西田 豊明, 教授 石田 亨, 講師 吉井 和佳 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DGAM

La robotique développementale a entrepris, au courant des quinze dernières années,d’étudier les processus développementaux, similaires à ceux des systèmes biologiques,chez les robots. Le but est de créer des robots qui ont une enfance—qui rampent avant d’essayer de courir, qui jouent avant de travailler—et qui basent leurs décisions sur l’expérience de toute une vie, incarnés dans le monde réel.Dans ce contexte, cette thèse étudie l’exploration sensorimotrice—la découverte pour un robot de son propre corps et de son environnement proche—pendant les premiers stage du développement, lorsque qu’aucune expérience préalable du monde n’est disponible. Plus spécifiquement, cette thèse se penche sur comment générer une diversité d’effets dans un environnement inconnu. Cette approche se distingue par son absence de fonction de récompense ou de fitness définie par un expert, la rendant particulièrement apte à être intégrée sur des robots auto-suffisants.Dans une première partie, l’approche est motivée et le problème de l’exploration est formalisé, avec la définition de mesures quantitatives pour évaluer le comportement des algorithmes et d’un cadre architectural pour la création de ces derniers. Via l’examen détaillé de l’exemple d’un bras robot à multiple degrés de liberté, la thèse explore quelques unes des problématiques fondamentales que l’exploration sensorimotrice pose, comme la haute dimensionnalité et la redondance sensorimotrice. Cela est fait en particulier via la comparaison entre deux stratégies d’exploration: le babillage moteur et le babillage dirigé par les objectifs. Plusieurs algorithmes sont proposés tour à tour et leur comportement est évalué empiriquement, étudiant les interactions qui naissent avec les contraintes développementales, les démonstrations externes et les synergies motrices. De plus, parce que même des algorithmes efficaces peuvent se révéler terriblement inefficaces lorsque leurs capacités d’apprentissage ne sont pas adaptés aux caractéristiques de leur environnement, une architecture est proposée qui peut dynamiquement choisir la stratégie d’exploration la plus adaptée parmi un ensemble de stratégies. Mais même avec de bons algorithmes, l’exploration sensorimotrice reste une entreprise coûteuse—un problème important, étant donné que les robots font face à des contraintes fortes sur la quantité de données qu’ils peuvent extraire de leur environnement;chaque observation prenant un temps non-négligeable à récupérer. [...] À travers cette thèse, les contributions les plus importantes sont les descriptions algorithmiques et les résultats expérimentaux. De manière à permettre la reproduction et la réexamination sans contrainte de tous les résultats, l’ensemble du code est mis à disposition. L’exploration sensorimotrice est un mécanisme fondamental du développement des systèmes biologiques. La séparer délibérément des mécanismes d’apprentissage et l’étudier pour elle-même dans cette thèse permet d’éclairer des problèmes importants que les robots se développant seuls seront amenés à affronter. / Developmental robotics has begun in the last fifteen years to study robots that havea childhood—crawling before trying to run, playing before being useful—and that are basing their decisions upon a lifelong and embodied experience of the real-world. In this context, this thesis studies sensorimotor exploration—the discovery of a robot’s own body and proximal environment—during the early developmental stages, when no prior experience of the world is available. Specifically, we investigate how to generate a diversity of effects in an unknown environment. This approach distinguishes itself by its lack of user-defined reward or fitness function, making it especially suited for integration in self-sufficient platforms. In a first part, we motivate our approach, formalize the exploration problem, define quantitative measures to assess performance, and propose an architectural framework to devise algorithms. through the extensive examination of a multi-joint arm example, we explore some of the fundamental challenges that sensorimotor exploration faces, such as high-dimensionality and sensorimotor redundancy, in particular through a comparison between motor and goal babbling exploration strategies. We propose several algorithms and empirically study their behaviour, investigating the interactions with developmental constraints, external demonstrations and biologicallyinspired motor synergies. Furthermore, because even efficient algorithms can provide disastrous performance when their learning abilities do not align with the environment’s characteristics, we propose an architecture that can dynamically discriminate among a set of exploration strategies. Even with good algorithms, sensorimotor exploration is still an expensive proposition— a problem since robots inherently face constraints on the amount of data they are able to gather; each observation takes a non-negligible time to collect. [...] Throughout this thesis, our core contributions are algorithms description and empirical results. In order to allow unrestricted examination and reproduction of all our results, the entire code is made available. Sensorimotor exploration is a fundamental developmental mechanism of biological systems. By decoupling it from learning and studying it in its own right in this thesis, we engage in an approach that casts light on important problems facing robots developing on their own.

While an infant is learning to grasp, there are two key processes that she uses for leading a successful development. In the first process, infants use an intuitional approach where the hand is moved towards the object to create an initial contact regardless of the object properties. The contact is followed by a tactile grasping phase where the object is enclosed by the hand. This intuitive grasping behavior leads an grasping mechanism, which utilizes visual input and incorporates this into the grasp plan. The second process is called scaffolding, a guidance by stating how to accomplish the task or modifying its behaviors by interference. Infants pay attention to such guidance and understand the indication of important features of an object from 9 months of age. This supervision mechanism plays an important role for learning how to grasp certain objects in a proper way. To simulate these behavioral findings, a reaching and a tactile grasping controller was implemented on iCub humanoid robot which allowed it to reach an object from different directions, and enclose its fingers to cover the object. With these, a human-like grasp learning for iCub is proposed. Namely, the first step is an unsupervised learning where the robot is experimenting how to grasp objects. The second step is supervised learning phase where a caregiver modifies the end-effectors position when the robot is mistaken. By doing several experiments for two different grasping styles, we observe that the proposed methodology shows a better learning rate comparing to the scaffolding-only learning mechanism.

We propose a developmental framework that enables the robot to learn affordances through interaction with the environment in an unsupervised way and to use these affordances at different levels of robot control, ranging from reactive response to planning. Inspired from Developmental Psychology, the robot&rsquo / s discovery of action possibilities is realized in two sequential phases. In the first phase, the robot that initially possesses a limited number of basic actions and reflexes discovers new behavior primitives by exercising these actions and by monitoring the changes created in its initially crude perception system. In the second phase, the robot explores a more complicated environment by executing the discovered behavior primitives and using more advanced perception to learn further action possibilities. For this purpose, first, the robot discovers commonalities in action-effect experiences by finding effect categories, and then builds predictors for each behavior to map object features and behavior parameters into effect categories. After learning affordances through self-interaction and self-observation, the robot can make plans to achieve desired goals, emulate end states of demonstrated actions, monitor the plan execution and take corrective actions using the perceptual structures employed or discovered during learning. Mobile and manipulator robots were used to realize the proposed framework. Similar to infants, these robots were able to form behavior repertoires, learn affordances, and gain prediction capabilities. The learned affordances were shown to be relative to the robots, provide perceptual economy and encode general relations. Additionally, the affordance-based planning ability was verified in various tasks such as table cleaning and object transportation.

In autonomous robotics, there is still a trend to develop and tune controllers with highly explicit goals and environments in mind. However, this tuning means that these robotic models often lack the developmental and behavioral flexibility seen in biological organisms. The lack of flexibility in these controllers leaves the robot vulnerable to changes in environmental condition. Whereby any environmental change may lead to the behaviors of the robots becoming unsuitable or even dangerous. In this manuscript we look at a potential biologically plausible mechanism which may be used in robotic controllers in order to allow them to adapt to different environments. This mechanism consists of a hormone driven epigenetic mechanism which regulates a robot's internal environment in relation to its current environmental conditions. As we will show in our early chapters, this epigenetic mechanism allows an autonomous robot to rapidly adapt to a range of different environmental conditions. This adaption is achieved without the need for any explicit knowledge of the environment. Allowing a single architecture to adapt to a range of challenges and develop unique behaviors. In later chapters however, we find that this mechanism not only allows for regulation of short term behavior, but also long development. Here we show how this system permits a robot to develop in a way that is suitable for its current environment. Further during this developmental process we notice similarities to infant development, along with acquisition of unplanned skills and abilities. The unplanned developments appears to leads to the emergence of unplanned potential cognitive abilities such as object permanence, which we assess using a range of different real world tests.

Robotics agents are meant to play an increasingly larger role in our everyday lives. To be successfully integrated in our environment, robots will need to develop and display adaptive, robust, and socially suitable behaviours. To tackle these issues, the robotics research community has invested a considerable amount of efforts in modelling robotic architectures inspired by research on living systems, from ethology to developmental psychology. Following a similar approach, this thesis presents the research results of the modelling and experimental testing of robotic architectures based on affective and attachment bonds between young infants and their primary caregiver. I follow a bottom-up approach to the modelling of such bonds, examining how they can promote the situated development of an autonomous robot. Specifically, the models used and the results from the experiments carried out in laboratory settings and with naive users demonstrate the impact such affective bonds have on the learning outcomes of an autonomous robot and on the perception and behaviour of humans. This research leads to the emphasis on the importance of the interplay between the dynamics of the regulatory behaviours performed by a robot and the responsiveness of the human partner. The coupling of such signals and behaviours in an attachment-like dyad determines the nature of the outcomes for the robot, in terms of learning or the satisfaction of other needs. The experiments carried out also demonstrate of the attachment system can help a robot adapt its own social behaviour to that of the human partners, as infants are thought to do during their development.

Afin de pouvoir évoluer de manière autonome et sûre dans leur environnement, les robots doivent être capables d'en construire un modèle fiable et pertinent. Pour des tâches variées dans des environnements complexes, il est difficile de prévoir de manière exhaustive les capacités nécessaires au robot. Il est alors intéressant de doter les robots de mécanismes d'apprentissage leur donnant la possibilité de construire eux-mêmes des représentations adaptées à leur environnement. Se posent alors deux questions : quelle doit être la nature des représentations utilisées et par quels mécanismes peuvent-elles être apprises ? Nous proposons pour cela l'utilisation de l'hypothèse des sous-variétés afin de développer des architectures permettant de faire émerger une représentation symbolique de flux sensorimoteurs bruts. Nous montrons que le paradigme de l'apprentissage profond fournit des mécanismes appropriés à l'apprentissage autonome de telles représentations. Nous démontrons que l'exploitation de la nature multimodale des flux sensorimoteurs permet d'en obtenir une représentation symbolique pertinente. Dans un second temps, nous étudions le problème de l'évolution temporelle des stimuli. Nous discutons les défauts de la plupart des approches aujourd'hui utilisées et nous esquissons une approche à partir de laquelle nous approfondissons deux sous-problèmes. Dans une troisième partie, nous proposons des pistes de recherche pour permettre le passage des expériences de laboratoire à des environnements naturels. Nous explorons plus particulièrement la problématique de la curiosité artificielle dans des réseaux de neurones non supervisés. / This thesis studies the use of deep neural networks to learn high level representations from raw inputs on robots, based on the "manifold hypothesis".

This dissertation investigates two complementary ideas in the literature on machine learning and robotics--those of embodiment and intrinsic motivation--to address a unified framework for skill learning and knowledge acquisition. "Embodied" systems make use of structure derived directly from sensory and motor configurations for learning behavior. Intrinsically motivated systems learn by searching for native, hedonic value through interaction with the world. Psychological theories of intrinsic motivation suggest that there exist internal drives favoring open-ended cognitive development and exploration. I argue that intrinsically motivated, embodied systems can learn generalizable skills, acquire control knowledge, and form an epistemological understanding of the world in terms of behavioral affordances. I propose that the development of behavior results from the assembly of an agent's sensory and motor resources into state and action spaces that can be explored autonomously. I introduce an intrinsic reward function that can lead to the open-ended learning of hierarchical behavior. This behavior is factored into declarative "recipes" for patterned activity and common sense procedural strategies for implementing them in a variety of run-time contexts. These skills form a categorical basis for the robot to interpret and model its world in terms of the behavior it accords. Experiments conducted on a bimanual robot illustrate a progression of cumulative manipulation behavior addressing manual and visual skills. Such accumulation of skill over the long-term by a single robot is a novel contribution that has yet to be demonstrated in the literature.

Dans cette thèse, nous nous intéressons à l'étude d'un modèle dédié à l'exploration et à la structuration d'espaces d'apprentissage sensorimoteur pour des systèmes artificiels. Nous appuyons notre démarche sur les notions de corps et de développement propre, auxquelles se greffe un troisième processus dit motivationnel. Cette forme de curiosité artificielle se base sur le progrès en compétence et repose ainsi sur les contraintes physiques naturelles directement issues de l'encorporation de l'agent. L'objectif de la motivation est de réguler un développement à long terme, dédié à l'apprentissage de nouvelles compétences non prévues par le concepteur. Nous inscrivons nos travaux dans la continuité de l'approche du babillage sensorimoteur dans l'espace des buts, qui consiste à déterminer un ensemble de techniques permettant à un agent de générer, selon une métrique d'intérêt, une configuration sensorielle qu'il va essayer d'atteindre par des actions motrices. Nos contributions viennent améliorer et complexifier un modèle motivationnel existant, en proposant des alternatives au processus de structuration de l'espace d'exploration. Certaines de ces propositions théoriques ont été validées expérimentalement grâce à la plateforme FIMO, que nous avons développée dans cette optique, et qui est disponible en ligne. / In this thesis, we study a motivational model for artificial systems, which aims at exploring and structuring sensorimotor learning spaces. Our approach relies on some essential notions, including the body, the development, and the motivation. This particular kind of artificial curiosity is based on the competence or learning progress, and thus depends on the physical natural constraints originating from the agent's embodiment. We follow the Goal-Babbling Exploration approach which consists in determining a set of techniques allowing an agent to self-generate goals, i.e. sensory configurations, and try to reach them using motor actions. Our contributions improve the SAGG-RIAC motivational model, by proposing alternative ways of structuring the exploration of the goal space. Some of our contributions have been experimentally validated within the FIMO framework we developed to this purpose.