In this thesis I report the development of FARSA (Framework for Autonomous Robotics Simulation and Analysis), a simulation tool for the study of the interaction between language and action in cognitive robots and more in general for experiments in embodied cognitive science. Before presenting the tools, I will describe a series of experiments that involve simulated humanoid robots that acquire their behavioural and language skills autonomously through a trial-and-error adaptive process in which random variations of the free parameters of the robots’ controller are retained or discarded on the basis of their effect on the overall behaviour exhibited by the robot in interaction with the environment. More specifically the first series of experiments shows how the availability of linguistic stimuli provided by a caretaker, that indicate the elementary actions that need to be carried out in order to accomplish a certain complex action, facilitates the acquisition of the required behavioural capacity. The second series of experiments shows how a robot trained to comprehend a set of command phrases by executing the corresponding appropriate behaviour can generalize its knowledge by comprehending new, never experienced sentences, and by producing new appropriate actions. Together with their scientific relevance, these experiments provide a series of requirements that have been taken into account during the development of FARSA. The objective of this project is that to reduce the complexity barrier that currently discourages part of the researchers interested in the study of behaviour and cognition from initiating experimental activity in this area. FARSA is the only available tools that provide an integrated framework for carrying on experiments of this type, i.e. it is the only tool that provides ready to use integrated components that enable to define the characteristics of the robots and of the environment, the characteristics of the robots’ controller, and the characteristics of the adaptive process. Overall this enables users to quickly setup experiments, including complex experiments, and to quickly start collecting results.

Este trabalho apresenta e discute uma proposta de estratégia inédita para o problema de aprendizado de regras através de Sistemas Classificadores, aplicado à robótica móvel, utilizando um robô NOMAD 200. Esta estratégia tem como base, teorias de Algoritmos Genéticos e de Sistemas Classificadores, que são os paradigmas constituintes do núcleo da arquitetura implementada para o controle do robô. O aspecto diferencial desta abordagem é a inspiração em Algoritmos Genéticos com Idades, para permitir o uso e controle de uma população de tamanho variável. O sistema foi modelado observando-se características físicas do robô NOMAD 200 e sendo constituído por módulos de gerenciamento de memória, reprodução, controle da população e execução. A memória se apresenta como uma base de regras de produção; o módulo de reprodução incorpora um AG tradicional, com operadores de seleção, cruzamento e mutação; o controle populacional permite o uso de população de tamanho variável, através do de índices de usabilidade e similaridade das regras com as situações confrontadas pelo robô; por fim, o módulo de execução é responsável pela interação do robô com o ambiente, realizando leitura dos sensores e ações pelos atuadores e, quando necessário, ativar funções de segurança para preservar a integridade física do robô. Para dar sustentabilidade à proposta, esta foi validada através de vários experimentos, realizados em ambientes simulados e em um ambiente real, com um robô NOMAD 200, em diferentes cenários. Os ambientes testados variam desde ambientes esparsos até labirintos com obstáculos e paredes ortogonais entre si. Para cada experimento são apresentados os resultados e respectiva análise de dados. Foram realizadas análises criteriosas no comportamento da população, observando seu crescimento e idade média, bem como os eventos ocorridos no processo de aprendizado, para certificar as características a que se propõe esta abordagem. A principal contribuição deste trabalho é o uso da "IDADE" e II"CSABILIDADE" em um sistema baseado em SC. A usabilidade substitui o atributo de energia e respectivos cálculos do SC tradicional, no processo de escolha das regras, simplificando a implementação. Além disso, pode ser utilizado como índice de ajuste, para que possam ser usadas técnicas convencionais de seleção. A idade é responsável por preservar ou eliminar os indivíduos da população, através de estratégias de penalização e recompensa, possibilitando manter uma população de regras de tamanho variável, permitindo, ainda, manter a diversidade genética na população e evitar a sua homogenização, bem como isentar o modelador do sistema da definição destes parâmetros. / In this work, we propose a new strategy to the problem of learning rules in a Evolutionary System that is applied for mobile robotics using a NOMAD 200 robot. This strategy is based on Genetic AIgoriths and Classifier Systems theories, which are the paradigms of the implemented architecture core for robot controI. The unique feature of this approach is the inspiration on Genetic AIgorithms with Ages. This feature allows the algorithm to make use of a controlled variable size population. The system was designed respecting the physical features of the ~OMAD 200 robot. It is composed by modules of memory, reproduction, populational control and execution. The memory is the base for production rules. The reproduction module is a conventional GA, with operators for selection, crossover and mutation. The population control allows the use of a variable size population, based on the usability and the similarity of the rules on the situations presented to the robot. Finally, the execution module is responsable for the interaction between the robot and the environment, making the sensors reading and action application from the actuators and, if necessary, activating the security functions to preserve the physical integrity of the robot. To give support to the proposal, it was validated through several experiments, performed both in a simulated environment and in a real NOMAD 200 robot, in several cenarios. The environments used in the experiments ranged from open spaces to labyrinths with obstacles and ortogonal walls. Vle present the results and data analysis for each one of the experiments. AIso, the population behavior is analysed, by the observation of his growing and average age and the events occurred during the learning process, to confirm the features of these approach. The main contribution of this work is the use of "AGE"and ""CSABILITY"in a CS based system. The usability replaces the strength attribute and respective calculations necessary in the process of choosing rules in traditional CS. Because of this change, our solution is simpler to implement than traditional CS systems. Besides that, the usability can be used as fitness value, making possible the use of conventional selection techniques. The Age is responsible for the decision of to preserve or to elliminate individuaIs from the population. The choose of individuaIs is done by a penalty and reward strategy, which permits a variable size population of rules with genetic diversity and avoid the population's homogenization. The use of the age for decision making aIso preserves the system developer from the task of defining these parameters.

Este trabalho apresenta e discute uma proposta de estratégia inédita para o problema de aprendizado de regras através de Sistemas Classificadores, aplicado à robótica móvel, utilizando um robô NOMAD 200. Esta estratégia tem como base, teorias de Algoritmos Genéticos e de Sistemas Classificadores, que são os paradigmas constituintes do núcleo da arquitetura implementada para o controle do robô. O aspecto diferencial desta abordagem é a inspiração em Algoritmos Genéticos com Idades, para permitir o uso e controle de uma população de tamanho variável. O sistema foi modelado observando-se características físicas do robô NOMAD 200 e sendo constituído por módulos de gerenciamento de memória, reprodução, controle da população e execução. A memória se apresenta como uma base de regras de produção; o módulo de reprodução incorpora um AG tradicional, com operadores de seleção, cruzamento e mutação; o controle populacional permite o uso de população de tamanho variável, através do de índices de usabilidade e similaridade das regras com as situações confrontadas pelo robô; por fim, o módulo de execução é responsável pela interação do robô com o ambiente, realizando leitura dos sensores e ações pelos atuadores e, quando necessário, ativar funções de segurança para preservar a integridade física do robô. Para dar sustentabilidade à proposta, esta foi validada através de vários experimentos, realizados em ambientes simulados e em um ambiente real, com um robô NOMAD 200, em diferentes cenários. Os ambientes testados variam desde ambientes esparsos até labirintos com obstáculos e paredes ortogonais entre si. Para cada experimento são apresentados os resultados e respectiva análise de dados. Foram realizadas análises criteriosas no comportamento da população, observando seu crescimento e idade média, bem como os eventos ocorridos no processo de aprendizado, para certificar as características a que se propõe esta abordagem. A principal contribuição deste trabalho é o uso da "IDADE" e II"CSABILIDADE" em um sistema baseado em SC. A usabilidade substitui o atributo de energia e respectivos cálculos do SC tradicional, no processo de escolha das regras, simplificando a implementação. Além disso, pode ser utilizado como índice de ajuste, para que possam ser usadas técnicas convencionais de seleção. A idade é responsável por preservar ou eliminar os indivíduos da população, através de estratégias de penalização e recompensa, possibilitando manter uma população de regras de tamanho variável, permitindo, ainda, manter a diversidade genética na população e evitar a sua homogenização, bem como isentar o modelador do sistema da definição destes parâmetros. / In this work, we propose a new strategy to the problem of learning rules in a Evolutionary System that is applied for mobile robotics using a NOMAD 200 robot. This strategy is based on Genetic AIgoriths and Classifier Systems theories, which are the paradigms of the implemented architecture core for robot controI. The unique feature of this approach is the inspiration on Genetic AIgorithms with Ages. This feature allows the algorithm to make use of a controlled variable size population. The system was designed respecting the physical features of the ~OMAD 200 robot. It is composed by modules of memory, reproduction, populational control and execution. The memory is the base for production rules. The reproduction module is a conventional GA, with operators for selection, crossover and mutation. The population control allows the use of a variable size population, based on the usability and the similarity of the rules on the situations presented to the robot. Finally, the execution module is responsable for the interaction between the robot and the environment, making the sensors reading and action application from the actuators and, if necessary, activating the security functions to preserve the physical integrity of the robot. To give support to the proposal, it was validated through several experiments, performed both in a simulated environment and in a real NOMAD 200 robot, in several cenarios. The environments used in the experiments ranged from open spaces to labyrinths with obstacles and ortogonal walls. Vle present the results and data analysis for each one of the experiments. AIso, the population behavior is analysed, by the observation of his growing and average age and the events occurred during the learning process, to confirm the features of these approach. The main contribution of this work is the use of "AGE"and ""CSABILITY"in a CS based system. The usability replaces the strength attribute and respective calculations necessary in the process of choosing rules in traditional CS. Because of this change, our solution is simpler to implement than traditional CS systems. Besides that, the usability can be used as fitness value, making possible the use of conventional selection techniques. The Age is responsible for the decision of to preserve or to elliminate individuaIs from the population. The choose of individuaIs is done by a penalty and reward strategy, which permits a variable size population of rules with genetic diversity and avoid the population's homogenization. The use of the age for decision making aIso preserves the system developer from the task of defining these parameters.

In evolutionary robotics there has been research about the pursuit problem with different numbers of predators and prey: (i) one predator and one prey, (ii) many predators against one prey, and (iii) many predators against many prey. However, these different experiments are only involving food chains with two populations (two trophic levels). This dissertation uses three trophic levels to investigate if individuals in the middle trophic level perform equally or better than those that are been evolved in a two trophic level environment. The investigation was done in a simulator called YAKS. A statistical analysis was conducted to evalutate the results. The result indicated that a robot with two tasks gets better at hunting and evading than robots with one task (either hunt or evade). Robots from the middle trophic level that are moving in the same direction as the camera is facing, were the best predators and prey. This dissertation is a step towards more complex and animal-like behaviours of robots.

This dissertation evaluates evolutionary methods for evolving cooperative teams of robots. Cooperative robotics is a challenging research area in the field of artificial intelligence. Individual and autonomous robots may by cooperation enhance their performance compared to what they can achieve separately. The challenge of cooperative robotics is that performance relies on interactions between robots. The interactions are not always fully understood, which makes the designing process of hardware and software systems complex. Robotic soccer, such as the RoboCup competitions, offers an unpredictable dynamical environment for competing robot teams that encourages research of these complexities. Instead of trying to solve these problems by designing and implement the behavior, the robots can learn how to behave by evolutionary methods. For this reason, this dissertation evaluates evolution of neural controllers for a team of two robots in a competitive soccer environment. The idea is that evolutionary methods may be a solution to the complexities of creating cooperative robots. The methods used in the experiments are influenced by research of evolutionary algorithms with single autonomous robots and on robotic soccer. The results show that robot teams can evolve to a form of cooperative behavior with simple reactive behavior by relying on self-adaptation with little supervision and human interference.

In the recent years a growing interest in Cognitive Science has been directed to the cognitive role of the agent's ability to predict the consequences of their actions, without actual engagement with their environment. The creation of an experimental model for Hesslow's simulation hypothesis, based on the use of a simulated adaptive agent and the methods of evolutionary robotics within the general perspective of radical connectionism, is reported in this dissertation. A hierarchical architecture consisting of a mixture of (recurrent) experts is investigated in order to test its ability to produce an 'inner world', functional stand-in for the agent's interactions with its environment. Such a mock world is expected to be rich enough to sustain 'blind navigation', which means navigation based solely on the agent's own internal predictions. The results exhibit the system's vivid internal dynamics, its critical sensitivity to a high number of parameters and, finally, a discrepancy with the declared goal of blind navigation. However, given the dynamical complexity of the system, further analysis and testing appear necessary.

In this work two novel frameworks, one using evolutionary algorithms and another using Reinforcement Learning for training Spiking Neural Networks are proposed and analyzed. A novel multi-agent evolutionary robotics (ER) based framework, inspired by competitive evolutionary environments in nature, is demonstrated for training Spiking Neural Networks (SNN). The weights of a population of SNNs along with morphological parameters of bots they control in the ER environment are treated as phenotypes. Rules of the framework select certain bots and their SNNs for reproduction and others for elimination based on their efficacy in capturing food in a competitive environment. While the bots and their SNNs are given no explicit reward to survive or reproduce via any loss function, these drives emerge implicitly as they evolve to hunt food and survive within these rules. Their efficiency in capturing food as a function of generations exhibit the evolutionary signature of punctuated equilibria. Two evolutionary inheritance algorithms on the phenotypes, Mutation and Crossover with Mutation along with their variants, are demonstrated. Performances of these algorithms are compared using ensembles of 100 experiments for each algorithm. We find that one of the Crossover with Mutation variants promotes 40% faster learning in the SNN than mere Mutation with a statistically significant margin. Along with an evolutionary approach to training SNNs, we also describe a novel Reinforcement Learning(RL) based framework using the Proximal Policy Optimization to train a SNN for an image classification task. The experiments and results of the framework are then discussed highlighting future direction of the work.

La coopération est un comportement présent en abondance dans le vivant et central à la plupart des transitions majeures en évolution. Pourtant, son évolution est difficile à expliquer. En particulier, l'origine des comportements de coopération mutualiste, qui sont bénéfiques à tous les participants, à été peu étudiés. Dans cette thèse, nous nous intéressons au rôle de la coordination dans l'évolution de la coopération mutualiste à l'aide d'outils en robotique évolutionniste. Dans un premier temps, nous modélisons en robotique évolutionniste l'évolution de la coopération dans une tâche de chasse collective afin d'étudier l'influence des mécanismes sous-jacent aux comportements de coordination sur l'évolution de la coopération. Nous montrons notamment la nécessité d'utiliser des modèles qui considèrent les aspects plus pratiques du comportement. Dans un second temps, nous nous intéressons à la conception de robots coopératifs à l'aide de la robotique évolutionniste. Nous étudions notamment l'influence de la composition génétique des groupes de robots sur l'efficacité des comportements de coordination. Cette thèse contribue donc de deux manières différentes au sujet général de l'évolution de la coopération : par de la modélisation de problèmes biologiques et la conception de robots collectifs. / Cooperation is a behaviour that is prevalent in living beings and is central to most of the major transitions in evolution. Yet explaining its origin is a challenge. In particular, the evolution of mutualistic behaviours, where every individual benefits from this behaviour, has been relatively ignored. In this thesis, we are interested in the role of coordination in the evolution of mutualistic cooperation. To that end, we use tools from evolutionary robotics. First, we model the evolution of cooperation in a collective hunting task with evolutionary robotics. We want to study the influence of the underlying mechanisms of coordination behaviours on the evolution of cooperation. We reveal that it is necessary to consider modeling techniques that account for the more practical aspects of behaviours. Then, we are interested in the design of cooperative robots with evolutionary robotics. We study the influence of genetic team composition on the efficiency of coordination behaviours. Therefore, this thesis contributes to the general subject of the evolution of cooperation in two different manners: modeling biological problems and designing collective robots.

No description available.

The work presented herein focuses on the design of distributed autonomous controllers for collective behaviour of Micro-unmanned Aerial Vehicles (MAVs). Two alternative approaches to this topic are introduced: one based upon the Evolutionary Robotics (ER) paradigm, the other one upon flocking principles. Three computer simulators have been developed in order to carry out the required experiments, all of them having their focus on the modelling of fixed-wing aircraft flight dynamics. The employment of fixed-wing aircraft rather than the omni-directional robots typically employed in collective robotics significantly increases the complexity of the challenges that an autonomous controller has to face. This is mostly due to the strict motion constraints associated with fixed-wing platforms, that require a high degree of accuracy by the controller. Concerning the ER approach, the experimental setups elaborated have resulted in controllers that have been evolved in simulation with the following capabilities: (1) navigation across unknown environments, (2) obstacle avoidance, (3) tracking of a moving target, and (4) execution of cooperative and coordinated behaviours based on implicit communication strategies. The design methodology based upon flocking principles has involved tests on computer simulations and subsequent experimentation on real-world robotic platforms. A customised implementation of Reynolds’ flocking algorithm has been developed and successfully validated through flight tests performed with the swinglet MAV. It has been notably demonstrated how the Evolutionary Robotics approach could be successfully extended to the domain of fixed-wing aerial robotics, which has never received a great deal of attention in the past. The investigations performed have also shown that complex and real physics-based computer simulators are not a compulsory requirement when approaching the domain of aerial robotics, as long as proper autopilot systems (taking care of the ”reality gap” issue) are used on the real robots.