The work presented herein focuses on the design of distributed autonomous controllers for collective behaviour of Micro-unmanned Aerial Vehicles (MAVs). Two alternative approaches to this topic are introduced: one based upon the Evolutionary Robotics (ER) paradigm, the other one upon flocking principles. Three computer simulators have been developed in order to carry out the required experiments, all of them having their focus on the modelling of fixed-wing aircraft flight dynamics. The employment of fixed-wing aircraft rather than the omni-directional robots typically employed in collective robotics significantly increases the complexity of the challenges that an autonomous controller has to face. This is mostly due to the strict motion constraints associated with fixed-wing platforms, that require a high degree of accuracy by the controller. Concerning the ER approach, the experimental setups elaborated have resulted in controllers that have been evolved in simulation with the following capabilities: (1) navigation across unknown environments, (2) obstacle avoidance, (3) tracking of a moving target, and (4) execution of cooperative and coordinated behaviours based on implicit communication strategies. The design methodology based upon flocking principles has involved tests on computer simulations and subsequent experimentation on real-world robotic platforms. A customised implementation of Reynolds’ flocking algorithm has been developed and successfully validated through flight tests performed with the swinglet MAV. It has been notably demonstrated how the Evolutionary Robotics approach could be successfully extended to the domain of fixed-wing aerial robotics, which has never received a great deal of attention in the past. The investigations performed have also shown that complex and real physics-based computer simulators are not a compulsory requirement when approaching the domain of aerial robotics, as long as proper autopilot systems (taking care of the ”reality gap” issue) are used on the real robots.

The work accomplished in this thesis concerns development of an autonomous machine cognition system. The proposed solution reposes on the assumption that it is the curiosity which motivates a cognitive system to acquire new knowledge. Further, two distinct kinds of curiosity are identified in conformity to human cognitive system. On this I build a two level cognitive architecture. I identify its lower level with the perceptual saliency mechanism, while the higher level performs knowledge acquisition from observation and interaction with the environment. This thesis brings the following contribution: A) Investigation of the state of the art in autonomous knowledge acquisition. B) Realization of a lower cognitive level in the ensemble of the mentioned system, which is realizing the perceptual curiosity mechanism through a novel fast, real-world robust algorithm for salient object detection and learning. C) Realization of a higher cognitive level through a general framework for knowledge acquisition from observation and interaction with the environment including humans. Based on the epistemic curiosity, the high-level cognitive system enables a machine (e.g. a robot) to be itself the actor of its learning. An important consequence of this system is the possibility to confer high level multimodal cognitive capabilities to robots to increase their autonomy in real-world environment (human environment). D) Realization of the strategy proposed in the context of autonomous robotics. The studies and experimental validations done had confirmed notably that our approach allows increasing the autonomy of robots in real-world environment

&ldquo / Developmental robotics&rdquo / proposes that, instead of trying to build a robot that shows intelligence once and for all, what one must do is to build robots that can develop. A robot should go through cognitive development just like an animal baby does. These robots should be equipped with behaviors that are simple but enough to bootstrap the system. Then, as the robot interacts with its environment, it should display increasingly complex behaviors. Studies in developmental psychology and neurophysiology provide support for the view that, the animals start with innate simple behaviors, and develop more complex behaviors through the differentiation, sequencing, and combination of these primitive behaviors. In this thesis, we propose such a development scheme for a mobile robot. J.J. Gibson&#039 / s concept of &ldquo / affordances&rdquo / provides the basis of this development scheme, and we use a formalization of affordances to make the robot learn about the dynamics of its interactions with its environment. We show that an autonomous robot can start with pre-coded primitive behaviors, and as it executes its behaviors randomly in an environment, it can learn the affordance relations between the environment and its behaviors. We then present two ways of using these learned structures, in achieving more complex, voluntary behaviors. In the first case, the robot still uses its pre-coded primitive behaviors only, but the sequencing of these are such that new more complex behaviors emerge. In the second case, the robot uses its pre-coded primitive behaviors to create new behaviors.

Un robot autonome collaborant avec des humains doit être capable d'apprendre à se déplacer et à manipuler des objets dans la même tâche. Dans une approche classique, on considère des modules fonctionnels indépendants gérant les différents aspects de la tâche (navigation, contrôle du bras...). A l'opposé, l'objectif de cette thèse est de montrer que l'apprentissage de tâches de natures différentes peut être abordé comme un problème d'apprentissage d'attracteurs sensorimoteurs à partir d'un petit nombre de structures non spécifiques à une tâche donnée. Nous avons donc proposé une architecture qui permet l'apprentissage et l'encodage d'attracteurs pour réaliser aussi bien des tâches de navigation que de contrôle d'un bras.Comme point de départ, nous nous sommes appuyés sur un modèle inspiré des cellules de lieu pour la navigation d'un robot autonome. Des apprentissages en ligne et interactifs de couples lieu/action sont suffisants pour faire émerger des bassins d'attraction permettant à un robot autonome de suivre une trajectoire. En interagissant avec le robot, on peut corriger ou orienter son comportement. Les corrections successives et leur encodage sensorimoteur permettent de définir le bassin d'attraction de la trajectoire. Ma première contribution a été d'étendre ce principe de construction d'attracteurs sensorimoteurs à un contrôle en impédance pour un bras robotique. Lors du maintien d'une posture proprioceptive, les mouvements du bras peuvent être corrigés par une modification en-ligne des commandes motrices exprimées sous la forme d'activations musculaires. Les attracteurs moteurs résultent alors des associations simples entre l'information proprioceptive du bras et ces commandes motrices. Dans un second temps, j'ai montré que le robot pouvait apprendre des attracteursvisuo-moteurs en combinant les informations proprioceptives et visuelles. Le contrôle visuo-moteur correspond à un homéostat qui essaie de maintenir un équilibre entre ces deux informations. Dans le cas d'une information visuelle ambiguë, le robot peut percevoir un stimulus externe (e.g. la main d'un humain) comme étant sa propre pince. Suivant le principe d'homéostasie, le robot agira pour réduire l'incohérence entre cette information externe et son information proprioceptive. Il exhibera alors un comportement d'imitation immédiate des gestes observés. Ce mécanisme d'homéostasie, complété par une mémoire des séquences observées et l'inhibition des actions durant l'observation, permet au robot de réaliser des imitations différées et d'apprendre par observation. Pour des tâches plus complexes, nous avons aussi montré que l'apprentissage de transitions peut servir de support pour l'apprentissage de séquences de gestes, comme c'était le cas pour l'apprentissage de cartes cognitives en navigation. L'utilisation de contextes motivationnels permet alors le choix entre les différentes séquences apprises.Nous avons ensuite abordé le problème de l'intégration dans une même architecture de comportements impliquant une navigation visuomotrice et le contrôle d'un bras robotique pour la préhension d'objets. La difficulté est de pouvoir synchroniser les différentes actions afin que le robot agisse de manière cohérente. Les comportements erronés du robot sont détectés grâce à l'évaluation des actions proposées par le modèle vis à vis des corrections imposées par le professeur humain. Un apprentissage de ces situations sous la forme de contextes multimodaux modulant la sélection d'action permet alors d'adapter le comportement afin que le robot reproduise la tâche désirée.Pour finir, nous présentons les perspectives de ce travail en terme de contrôle sensorimoteur, pour la navigation comme pour le contrôle d'un bras robotique, et son extension aux questions d'interface homme/robot. Nous insistons sur le fait que différents types d'imitation peuvent être le fruit des propriétés émergentes d'une architecture de contrôle sensorimotrice. / An autonomous robot collaborating with humans should be able to learn how to navigate and manipulate objects in the same task. In a classical approach, independent functional modules are considered to manage the different aspects of the task (navigation, arm control,...) . To the contrary, the goal of this thesis is to show that learning tasks of different kinds can be tackled by learning sensorimotor attractors from a few task nonspecific structures. We thus proposed an architecture which can learn and encode attractors to perform navigation tasks as well as arm control.We started by considering a model inspired from place-cells for navigation of autonomous robots. On-line and interactive learning of place-action couples can let attraction basins emerge, allowing an autonomous robot to follow a trajectory. The robot behavior can be corrected and guided by interacting with it. The successive corrections and their sensorimotor coding enables to define the attraction basin of the trajectory. My first contribution was to adapt this principle of sensorimotor attractor building for the impedance control of a robot arm. While a proprioceptive posture is maintained, the arm movements can be corrected by modifying on-line the motor command expressed as muscular activations. The resulting motor attractors are simple associations between the proprioceptive information of the arm and these motor commands. I then showed that the robot could learn visuomotor attractors by combining the proprioceptive and visual information with the motor attractors. The visuomotor control corresponds to a homeostatic system trying to maintain an equilibrium between the two kinds of information. In the case of ambiguous visual information, the robot may perceive an external stimulus (e.g. a human hand) as its own hand. According to the principle of homeostasis, the robot will act to reduce the incoherence between this external information and its proprioceptive information. It then displays a behavior of immediately observed gestures imitation. This mechanism of homeostasis, completed by a memory of the observed sequences and action inhibition capability during the observation phase, enables a robot to perform deferred imitation and learn by observation. In the case of more complex tasks, we also showed that learning transitions can be the basis for learning sequences of gestures, like in the case of cognitive map learning in navigation. The use of motivational contexts then enables to choose between different learned sequences.We then addressed the issue of integrating in the same architecture behaviors involving visuomotor navigation and robotic arm control to grab objects. The difficulty is to be able to synchronize the different actions so the robot act coherently. Erroneous behaviors of the robot are detected by evaluating the actions predicted by the model with respect to corrections forced by the human teacher. These situations can be learned as multimodal contexts modulating the action selection process in order to adapt the behavior so the robot reproduces the desired task.Finally, we will present the perspectives of this work in terms of sensorimotor control, for both navigation and robotic arm control, and its link to human robot interface issues. We will also insist on the fact that different kinds of imitation behavior can result from the emergent properties of a sensorimotor control architecture.

Les êtres humains définissent naturellement leur espace quotidien en unités discrètes. Par exemple, nous sommes capables d'identifier le lieu où nous sommes (e.g. le bureau 205) et sa catégorie (i.e. un bureau), sur la base de leur seule apparence visuelle. Les travaux récents en reconnaissance de lieux sémantiques, visent à doter les robots de capacités similaires. Ces unités, appelées "lieux sémantiques", sont caractérisées par une extension spatiale et une unité fonctionnelle, ce qui distingue ce domaine des travaux habituels en cartographie. Nous présentons nos travaux dans le domaine de la reconnaissance de lieux sémantiques. Ces derniers ont plusieurs originalités par rapport à l'état de l'art. Premièrement, ils combinent la caractérisation globale d'une image, intéressante car elle permet de s'affranchir des variations locales de l'apparence des lieux, et les méthodes basées sur les mots visuels, qui reposent sur la classification non-supervisée de descripteurs locaux. Deuxièmement, et de manière intimement reliée, ils tirent parti du flux d'images fourni par le robot en utilisant des méthodes bayésiennes d'intégration temporelle. Dans un premier modèle, nous ne tenons pas compte de l'ordre des images. Le mécanisme d'intégration est donc particulièrement simple mais montre des difficultés à repérer les changements de lieux. Nous élaborons donc plusieurs mécanismes de détection des transitions entre lieux qui ne nécessitent pas d'apprentissage supplémentaire. Une deuxième version enrichit le formalisme classique du filtrage bayésien en utilisant l'ordre local d'apparition des images. Nous comparons nos méthodes à l'état de l'art sur des tâches de reconnaissance d'instances et de catégorisation, en utilisant plusieurs bases de données. Nous étudions l'influence des paramètres sur les performances et comparons les différents types de codage employés sur une même base.Ces expériences montrent que nos méthodes sont supérieures à l'état de l'art, en particulier sur les tâches de catégorisation. / Human beings naturally organize their space as composed of discrete units. Those units, called "semantic places", are characterized by their spatial extend and their functional unity. Moreover, we are able to quickly recognize a given place (e.g. office 205) and its category (i.e. an office), solely on their visual appearance. Recent works in semantic place recognition seek to endow the robot with similar capabilities. Contrary to classical localization and mapping work, this problem is usually tackled as a supervised learning problem. Our contributions are two fold. First, we combine global image characterization, which captures the global organization of the image, and visual words methods which are usually based unsupervised classification of local signatures. Our second but closely related, contribution is to use several images for recognition by using Bayesian methods for temporal integration. Our first model don't use the natural temporal ordering of images. Temporal integration is very simple but has difficulties when the robot moves from one place to another.We thus develop several mechanisms to detect place transitions. Those mechanisms are simple and don't require additional learning. A second model augment the classical Bayesian filtering approach by using the local order among images. We compare our methods to state-of-the-art algorithms on place recognition and place categorization tasks.We study the influence of system parameters and compare the different global characterization methods on the same dataset. These experiments show that our approach while being simple leads to better results especially on the place categorization task.

Le travail effectué lors de cette thèse concerne le développement d'un système cognitif artificiel autonome. La solution proposée repose sur l'hypothèse que la curiosité est une source de motivation d'un système cognitif dans le processus d'acquisition des nouvelles connaissances. En outre, deux types distincts de curiosité ont été identifiés conformément au système cognitif humain. Sur ce principe, une architecture cognitive à deux niveaux a été proposée. Le bas-niveau repose sur le principe de la saillance perceptive, tandis que le haut-niveau réalise l'acquisition des connaissances par l'observation et l'interaction avec l'environnement. Cette thèse apporte les contributions suivantes : A) Un état de l'art sur l'acquisition autonome de connaissance. B) L'étude, la conception et la réalisation d'un système cognitif bas-niveau basé sur le principe de la curiosité perceptive. L'approche proposée repose sur la saillance visuelle réalisée grâce au développement d'un algorithme rapide et robuste permettant la détection et l'apprentissage d'objets saillants. C) La conception d'un système cognitif haut-niveau, basé sur une approche générique, permettant l'acquisition de connaissance à partir de l'observation et de l'interaction avec son environnent (y compris avec les êtres humains). Basé sur la curiosité épistémique, le système cognitif haut-niveau développé permet à une machine (par exemple un robot) de devenir l'acteur de son propre apprentissage. Une conséquence substantielle d'un tel système est la possibilité de conférer des capacités cognitives haut-niveau multimodales à des robots pour accroître leur autonomie dans un environnement réel (environnement humain). D) La mise en œuvre de la stratégie proposée dans le cadre de la robotique autonome. Les études et les validations expérimentales réalisées ont notamment confirmé que notre approche permet d'accroître l'autonomie des robots dans un environnement réel / The work accomplished in this thesis concerns development of an autonomous machine cognition system. The proposed solution reposes on the assumption that it is the curiosity which motivates a cognitive system to acquire new knowledge. Further, two distinct kinds of curiosity are identified in conformity to human cognitive system. On this I build a two level cognitive architecture. I identify its lower level with the perceptual saliency mechanism, while the higher level performs knowledge acquisition from observation and interaction with the environment. This thesis brings the following contribution: A) Investigation of the state of the art in autonomous knowledge acquisition. B) Realization of a lower cognitive level in the ensemble of the mentioned system, which is realizing the perceptual curiosity mechanism through a novel fast, real-world robust algorithm for salient object detection and learning. C) Realization of a higher cognitive level through a general framework for knowledge acquisition from observation and interaction with the environment including humans. Based on the epistemic curiosity, the high-level cognitive system enables a machine (e.g. a robot) to be itself the actor of its learning. An important consequence of this system is the possibility to confer high level multimodal cognitive capabilities to robots to increase their autonomy in real-world environment (human environment). D) Realization of the strategy proposed in the context of autonomous robotics. The studies and experimental validations done had confirmed notably that our approach allows increasing the autonomy of robots in real-world environment

Cette thèse s'inscrit dans le cadre du projet IDEAL (Implementing DevelopmentAl Learning) financé par l'Agence Nationale de la Recherche (ANR). La capacité à percevoir, mémoriser et interpréter l'environnement qui nous entoure est une capacité vitale que l'on retrouve chez de nombreux êtres vivants. Cette capacité leur permet de générer des comportements adaptés à leur contexte, ou d'échapper à un prédateur sorti de leur champ de vision. L'objectif de cette thèse consiste à doter un agent artificiel de cette capacité. Nous proposons un modèle théorique permettant à un agent artificiel de générer des connaissances exploitables des éléments constituant son environnement et une structure reflétant l'espace. Ce modèle est basé sur la théorie de la contingence sensorimotrice, et implémente une forme de motivation intrinsèque. En effet, ce modèle débute avec un ensemble de structure indivisibles, appelées interactions, caractérisant les échanges entre l'agent et son environnement. L'apprentissage des connaissances est développemental et émerge de l'interaction entre l'agent et son environnement, sans qu'aucune intervention extérieure (récompense), ne soit nécessaire. Notre modèle propose des mécanismes permettant d'organiser et d'exploiter ces connaissances émergentes dans le but de générer des comportements. Nous proposons des implémentations de ce modèle pour démontrer l'émergence d'une connaissance à partir de l'interaction entre l'agent et son environnement, et les comportements qui émergent de cette connaissance / This thesis is a part of the IDEAL project (Implementing DEvelopmentAl Learning) funded by the Agence Nationale de la Recherche (ANR). The ability of perceiving, memorizing and interpreting the surrounding environment is a vital ability found in numerous living beings. This ability allows them to generate context adapted behaviors, or escaping from a predator that escape from their sensory system. The objective of this thesis consists in implementing such a capacity in artificial agents. We propose a theoretical model that allows artificial agent to generate a usable knowledge of elements that compose its environment and a structure able to characterize the structure of surrounding space. This model is based on the sensorimotor contingency theory, and implements a form of intrinsic motivation. Indeed, this model begin with a set of indivisible structures, called interactions, that characterize the interaction possibilities between the agent and its environment. The learning is developmental and emerges from the interaction that occurs between the agent and the environment, without the need of any external intervention (like reward). Our model propose a set of mechanisms that allow to organize and exploit emerging knowledge in order to generate behaviors. We propose implementations of our model to demonstrate the emerging knowledge based on agent-environment interaction, and behaviors that can emerge from this knowledge

Les Processus Décisionnels de Markov Partiellement Observables (PDMPOs) permettent de modéliser facilement lesproblèmes probabilistes de décision séquentielle dans l'incertain. Lorsqu'il s'agit d'une mission robotique, lescaractéristiques du robot et de son environnement nécessaires à la définition de la mission constituent le système. Son étatn'est pas directement visible par l'agent (le robot). Résoudre un PDMPO revient donc à calculer une stratégie qui remplit lamission au mieux en moyenne, i.e. une fonction prescrivant les actions à exécuter selon l'information reçue par l'agent. Cetravail débute par la mise en évidence, dans le contexte robotique, de limites pratiques du modèle PDMPO: ellesconcernent l'ignorance de l'agent, l'imprécision du modèle d'observation ainsi que la complexité de résolution. Unhomologue du modèle PDMPO appelé pi-PDMPO, simplifie la représentation de l'incertitude: il vient de la Théorie desPossibilités Qualitatives qui définit la plausibilité des événements de manière qualitative, permettant la modélisation del'imprécision et de l'ignorance. Une fois les modèles PDMPO et pi-PDMPO présentés, une mise à jour du modèle possibilisteest proposée. Ensuite, l'étude des pi-PDMPOs factorisés permet de mettre en place un algorithme appelé PPUDD utilisantdes Arbres de Décision Algébriques afin de résoudre plus facilement les problèmes structurés. Les stratégies calculées parPPUDD, testées par ailleurs lors de la compétition IPPC 2014, peuvent être plus efficaces que celles des algorithmesprobabilistes dans un contexte d'imprécision ou de grande dimension. Cette thèse propose d'utiliser les possibilitésqualitatives dans le but d'obtenir des améliorations en termes de temps de calcul et de modélisation. / Partially Observable Markov Decision Processes (POMDPs) define a useful formalism to express probabilistic sequentialdecision problems under uncertainty. When this model is used for a robotic mission, the system is defined as the featuresof the robot and its environment, needed to express the mission. The system state is not directly seen by the agent (therobot). Solving a POMDP consists thus in computing a strategy which, on average, achieves the mission best i.e. a functionmapping the information known by the agent to an action. Some practical issues of the POMDP model are first highlightedin the robotic context: it concerns the modeling of the agent ignorance, the imprecision of the observation model and thecomplexity of solving real world problems. A counterpart of the POMDP model, called pi-POMDP, simplifies uncertaintyrepresentation with a qualitative evaluation of event plausibilities. It comes from Qualitative Possibility Theory whichprovides the means to model imprecision and ignorance. After a formal presentation of the POMDP and pi-POMDP models,an update of the possibilistic model is proposed. Next, the study of factored pi-POMDPs allows to set up an algorithmnamed PPUDD which uses Algebraic Decision Diagrams to solve large structured planning problems. Strategies computedby PPUDD, which have been tested in the context of the competition IPPC 2014, can be more efficient than those producedby probabilistic solvers when the model is imprecise or for high dimensional problems. This thesis proposes some ways ofusing Qualitative Possibility Theory to improve computation time and uncertainty modeling in practice.

Dynamic Collaborative Driving is the concept of decentralized multi-vehicle automated driving where vehicles form dynamic local area networks within which information is shared to build a dynamic data representation of the environment to improve road usage and safety. The vision is to have networks of cars spanning multiple lanes forming these dynamic networks so as to optimize traffic flow while maintaining safety as each vehicle travels to its destinations. A basic requirement of any vehicle participating in dynamic collaborative driving is longitudinal and lateral control. Without this capability, higher-level coordination is not possible. This thesis investigates the issue of the control of an automobile in the context of a Dynamic Collaborative Driving system. Each vehicle involved is considered a complex composite nonlinear system. Therefore a complex nonlinear model of the vehicle dynamics is formulated and serves as the control system design platform. Due to the nonlinear nature of the vehicle dynamics, a nonlinear approach to control is used to achieve longitudinal and lateral control of the vehicle. This novel approach combines the use of reinforcement learning: a modern machine learning technique, with adaptive control and preview control techniques. This thesis presents the design of both the longitudinal and lateral control systems which serves as a basis for Dynamic Collaborative Driving. The results of the reinforcement learning phase and the performance of the adaptive control systems for single automobile performance as well as the performance in a multi-vehicle platoon is presented.

Dynamic Collaborative Driving is the concept of decentralized multi-vehicle automated driving where vehicles form dynamic local area networks within which information is shared to build a dynamic data representation of the environment to improve road usage and safety. The vision is to have networks of cars spanning multiple lanes forming these dynamic networks so as to optimize traffic flow while maintaining safety as each vehicle travels to its destinations. A basic requirement of any vehicle participating in dynamic collaborative driving is longitudinal and lateral control. Without this capability, higher-level coordination is not possible. This thesis investigates the issue of the control of an automobile in the context of a Dynamic Collaborative Driving system. Each vehicle involved is considered a complex composite nonlinear system. Therefore a complex nonlinear model of the vehicle dynamics is formulated and serves as the control system design platform. Due to the nonlinear nature of the vehicle dynamics, a nonlinear approach to control is used to achieve longitudinal and lateral control of the vehicle. This novel approach combines the use of reinforcement learning: a modern machine learning technique, with adaptive control and preview control techniques. This thesis presents the design of both the longitudinal and lateral control systems which serves as a basis for Dynamic Collaborative Driving. The results of the reinforcement learning phase and the performance of the adaptive control systems for single automobile performance as well as the performance in a multi-vehicle platoon is presented.