[pt] Em diversos problemas encontrados na literatura, se faz necessária alguma coordenação entre os agentes para que a tarefa seja realizada de forma ótima. Entretanto, pode ser difícil a obtenção desta coordenação por conta da quantidade e características dos agentes, dinâmica do ambiente e/ou complexidade da tarefa. O objetivo principal deste estudo é propor um modelo que possa se adaptar a problemas heterogêneos de coordenação e de dimensões elevadas, com aprendizado autônomo e que tenha convergência satisfatória, o qual foi denominado Modelo de Neuro Co-Evolução com Inspiração Quântica (NCoQ). O modelo se utiliza dos paradigmas da física quântica e da co-evolução biológica, evoluindo concomitantemente sub-populações de indivíduos quânticos para obter ganhos de convergência. A representação dos indivíduos por pulsos quânticos consegue reduzir o número de indivíduos em cada população, além de ser a mais recomendada para a utilização de neuro-evolução por conta da representação real. Ressalta-se também a capacidade do modelo em obter de forma autônoma a melhor configuração de arquitetura para as redes neurais de cada agente, não exigindo do programador a escolha deste parâmetro. Foram propostos novos operadores quânticos de crossover e mutação que foram comparados na otimização de funções de diversas dimensões. Para testar o desempenho do modelo, foram desenvolvidas, em linguagem MATLAB, simulações para o problema presa predador, para o benchmark multi-rover de exploração de ambientes e uma simulação para cobertura telefônica. Foram feitas comparações com outros modelos neuro-evolutivos encontrados na literatura, tendo o modelo NCoQ apresentado os melhores resultados. / [en] Many problems in the literature require some coordination among agents so a specific task can be executed more efficiently. However, this coordination can be difficult because of the quantity and characteristics of the agents, environment dynamics and/or task complexity. The main contribution of this Thesis is the proposal of a model, called Quantum Inspired Neuro Co-Evolution (NCoQ), that can adapt to heterogeneous multi-agent problems in high dimensions utilizing self-learning and that has satisfactory convergence. The model is inspired in quantum physics and biological co-evolution paradigms and evolves concomitantly subpopulations of quantum individuals to get convergence gains. The representation of individuals for quantum functions is able to reduce the numbers of individuals in each population and it is the most recommended for real neuro-evolution representation. It s also important to point out the model capacity in self-finding the best architecture of the neural networks agents, not requiring an a priori definition of this parameter. New crossover and mutation quantum operators were also proposed and compared in functions optimization of multiple dimensions. To test the model performance, three MATLAB simulations were developed: prey-predator task, multi-rover task and cell phone coverage area simulation. Comparisons were made against others neuro-evolution models found in literature and the NCoQ model attained the best results.

In this thesis, the evolution of adaptive behaviour in artificial agents is studied. More specifically, two types of adaptive behaviours are studied: articulated and cognitive ones. Chapter 1 presents a general introduction together with a brief presentation of the research area of this thesis, its main goals and a brief overview of the experimental studies done, the results and conclusions obtained. On chapter 2, I briefly present some promising methods that automatically generate robot controllers and/or body plans and potentially could help in the development of adaptive robots. Among these methods I present in details evolutionary robotics, a method inspired on natural evolution, and the biological background regarding adaptive behaviours in biological organisms, which provided inspiration for the studies presented in this thesis. On chapter 3, I present a detailed study regarding the evolution of articulated behaviours, i.e., behaviours that are organized in functional sub-parts, and that are combined and used in a sequential and context-dependent way, regardless if there is a structural division in the robot controller or not. The experiments performed with a single goal task, a cleaning task, showed that it is possible to evolve articulated behaviours even in this condition and without structural division of the robot controller. Also the analysis of the results showed that this type of integrated modular behaviours brought performance advantages compared to structural divided controllers. Analysis of robots' behaviours helped to clarify that the evolution of this type of behaviour depended on the characteristics of the neural network controllers and the robot's sensorimotor capacities, that in turn defined the capacity of the robot to generate opportunity for actions, which in psychological literature is often called affordances. In chapter 4, a study seeking to understand the role of reactive strategies in the evolution of cognitive solutions, i.e. those capable of integrating information over time encoding it on internal states that will regulate the robot's behaviour in the future, is presented. More specifically I tried to understand whether the existence of sub-optimal reactive strategies prevent the development of cognitive solutions, or they can promote the evolution of solutions capable of combining reactive strategies and the use of internal information for solving a response delayed task, the double t-maze. The results obtained showed that reactive strategies capable of offloading cognitive work to the agent/environmental relation can promote, rather than prevent the evolution of solutions relying on internal information. The analysis of these results clarified how these two mechanisms interact producing a hybrid superior and robust solution for the delayed response task.

Orientador: Prof. Dr. Fabrício Olivetti de França / Dissertação (mestrado) - Universidade Federal do ABC, Programa de Pós-Graduação em Ciência da Computação, 2016. / No campo de Inteligência Artficial uma dasareas de interesse é a de criação de agentes inteligentes. Essa area de estudo tem o objetivo de construir um agente capaz de tomar decisões sem intervenção humana, para cumprir determinadas tarefas. Um desafio dessa área é manter o bom cumprimento da tarefa quando o agente enfrenta situações distintas do que ele observou durante a fase do aprendizado. Uma das possíveis aplicações desses agentesé na área de Jogos Eletrronicos, em que agentes autonomos podem ser programados para vencer os desafios projetados pelos programadores, com o objetivo de verificar a viabilidade e dificuldade de cada estagio. Da mesma forma, os jogos eletrronicos podem ser utilizados como plataforma para criação de novos algoritmos de aprendizado para agentes autonomos a serem aplicados em situações diversas. Este trabalho proproe um ambiente de teste chamado EvoMan, onde os oito inimigo finais do jogo eletronico MegaMan II 1 foram reproduzidos e estão disponíveis para treinar agentes controladores articiais de diferentes formas, simulando a dinâmica de um ambiente incerto. Os experimentos compreendem o uso de algoritmos de NeuroEvolução para encontrar conjuntos de agentes autônomos capazes de enfrentar os diversos desaos interpostos no ambiente proposto de forma generalista.

[pt] As redes neurais deep são modelos poderosos e flexíveis, que ganharam destaque na comunidade científica na última década. Para muitas tarefas, elas até superam o desempenho humano. Em geral, para obter tais resultados, um especialista despende tempo significativo para projetar a arquitetura neural, com longas sessões de tentativa e erro. Com isso, há um interesse crescente em automatizar esse processo. Novos métodos baseados em técnicas como aprendizado por reforço e algoritmos evolutivos foram apresentados como abordagens para o problema da busca de arquitetura neural (NAS - Neural Architecture Search), mas muitos ainda são algoritmos de alto custo computacional. Para reduzir esse custo, pesquisadores sugeriram limitar o espaço de busca, com base em conhecimento prévio. Os algoritmos evolutivos de inspiração quântica (AEIQ) apresentam resultados promissores em relação à convergência mais rápida. A partir dessa idéia, propõe-se o Q-NAS: um AEIQ para buscar redes deep através da montagem de subestruturas. O Q-NAS também pode evoluir alguns hiperparâmetros numéricos, o que é um primeiro passo para a automação completa. Experimentos com o conjunto de dados CIFAR-10 foram realizados a fim de analisar detalhes do Q-NAS. Para muitas configurações de parâmetros, foram obtidos resultados satisfatórios. As melhores acurácias no CIFAR-10 foram de 93,85 porcento para uma rede residual e 93,70 porcento para uma rede convolucional, superando modelos elaborados por especialistas e alguns métodos de NAS. Incluindo um esquema simples de parada antecipada, os tempos de evolução nesses casos foram de 67 dias de GPU e 48 dias de GPU, respectivamente. O Q-NAS foi aplicado ao CIFAR-100, sem qualquer ajuste de parâmetro, e obteve 74,23 porcento de acurácia, similar a uma ResNet com 164 camadas. Por fim, apresenta-se um estudo de caso com dados reais, no qual utiliza-se o Q-NAS para resolver a tarefa de classificação sísmica. Em menos de 8,5 dias de GPU, o Q-NAS gerou redes com 12 vezes menos pesos e maior acurácia do que um modelo criado especialmente para esta tarefa. / [en] Deep neural networks are powerful and flexible models that have gained the attention of the machine learning community over the last decade. For a variety of tasks, they can even surpass human-level performance. Usually, to reach these excellent results, an expert spends significant time designing the neural architecture, with long trial and error sessions. In this scenario, there is a growing interest in automating this design process. To address the neural architecture search (NAS) problem, authors have presented new methods based on techniques such as reinforcement learning and evolutionary algorithms, but the high computational cost is still an issue for many of them. To reduce this cost, researchers have proposed to restrict the search space, with the help of expert knowledge. Quantum-inspired evolutionary algorithms present promising results regarding faster convergence. Motivated by this idea, we propose Q-NAS: a quantum-inspired algorithm to search for deep networks by assembling substructures. Q-NAS can also evolve some numerical hyperparameters, which is a first step in the direction of complete automation. We ran several experiments with the CIFAR-10 dataset to analyze the details of the algorithm. For many parameter settings, Q-NAS was able to achieve satisfactory results. Our best accuracies on the CIFAR-10 task were 93.85 percent for a residual network and 93.70 percent for a convolutional network, overcoming hand-designed models, and some NAS works. Considering the addition of a simple early-stopping mechanism, the evolution times for these runs were 67 GPU days and 48 GPU days, respectively. Also, we applied Q-NAS to CIFAR-100 without any parameter adjustment, reaching an accuracy of 74.23 percent, which is comparable to a ResNet with 164 layers. Finally, we present a case study with real datasets, where we used Q-NAS to solve the seismic classification task. In less than 8.5 GPU days, Q-NAS generated networks with 12 times fewer weights and higher accuracy than a model specially created for this task.

While it is important for systems intended to enhance musical creativity to define and explore musical ideas conceived by individual users, many limit musical freedom by focusing on maintaining musical structure, thereby impeding the user's freedom to explore his or her individual style. This dissertation presents a comprehensive body of work that introduces a new musical representation that allows users to explore a space of musical rules that are created from their own melodies. This representation, called functional scaffolding for musical composition (FSMC), exploits a simple yet powerful property of multipart compositions: The pattern of notes and rhythms in different instrumental parts of the same song are functionally related. That is, in principle, one part can be expressed as a function of another. Music in FSMC is represented accordingly as a functional relationship between an existing human composition, or scaffold, and an additional generated voice. This relationship is encoded by a type of artificial neural network called a compositional pattern producing network (CPPN). A human user without any musical expertise can then explore how these additional generated voices should relate to the scaffold through an interactive evolutionary process akin to animal breeding. The utility of this insight is validated by two implementations of FSMC called NEAT Drummer and MaestroGenesis, that respectively help users tailor drum patterns and complete multipart arrangements from as little as a single original monophonic track. The five major contributions of this work address the overarching hypothesis in this dissertation that functional relationships alone, rather than specialized music theory, are sufficient for generating plausible additional voices. First, to validate FSMC and determine whether plausible generated voices result from the human-composed scaffold or intrinsic properties of the CPPN, drum patterns are created with NEAT Drummer to accompany several different polyphonic pieces. Extending the FSMC approach to generate pitched voices, the second contribution reinforces the importance of functional transformations through quality assessments that indicate that some partially FSMC-generated pieces are indistinguishable from those that are fully human. While the third contribution focuses on constructing and exploring a space of plausible voices with MaestroGenesis, the fourth presents results from a two-year study where students discuss their creative experience with the program. Finally, the fifth contribution is a plugin for MaestroGenesis called MaestroGenesis Voice (MG-V) that provides users a more natural way to incorporate MaestroGenesis in their creative endeavors by allowing scaffold creation through the human voice. Together, the chapters in this dissertation constitute a comprehensive approach to assisted music generation, enabling creativity without the need for musical expertise.

Computer-generated music composition programs have yet to produce creative, natural sounding music. To date, most approaches constrain the search space heuristically while ignoring the inherent structure of music over time. To address this problem, this thesis introduces NEAT Drummer, which evolves a special kind of artificial neural network (ANN) called compositional pattern producing networks (CPPNs) with the NeuroEvolution of Augmenting Topologies (NEAT) method for evolving increasingly complex structures. CPPNs in NEAT Drummer input existing human compositions and output an accompanying drum track. The existing musical parts form a scaffold i.e. support structure, for the drum pattern outputs, thereby exploiting the functional relationship of drums to musical parts (e.g. to lead guitar, bru:is, etc.) The results are convincing drum patterns that follow the contours of the original song, validating a new approach to computergenerated music composition.

Artificial Neural Networks (ANNs) have been comprehensively researched, both from a computer scientific perspective and with regard to their use for predictive modelling in a wide variety of applications including hydrology and the environment. Yet their adoption for live, real-time systems remains on the whole sporadic and experimental. A plausible hypothesis is that this may be at least in part due to their treatment heretofore as “black boxes” that implicitly contain something that is unknown, or even unknowable. It is understandable that many of those responsible for delivering Early Warning Systems (EWS) might not wish to take the risk of implementing solutions perceived as containing unknown elements, despite the computational advantages that ANNs offer. This thesis therefore builds on existing efforts to open the box and develop tools and techniques that visualise, analyse and use ANN weights and biases especially from the viewpoint of neural pathways from inputs to outputs of feedforward networks. In so doing, it aims to demonstrate novel approaches to self-improving predictive model construction for both regression and classification problems. This includes Neural Pathway Strength Feature Selection (NPSFS), which uses ensembles of ANNs trained on differing subsets of data and analysis of the learnt weights to infer degrees of relevance of the input features and so build simplified models with reduced input feature sets. Case studies are carried out for prediction of flooding at multiple nodes in urban drainage networks located in three urban catchments in the UK, which demonstrate rapid, accurate prediction of flooding both for regression and classification. Predictive skill is shown to reduce beyond the time of concentration of each sewer node, when actual rainfall is used as input to the models. Further case studies model and predict statutory bacteria count exceedances for bathing water quality compliance at 5 beaches in Southwest England. An illustrative case study using a forest fires dataset from the UCI machine learning repository is also included. Results from these model ensembles generally exhibit improved performance, when compared with single ANN models. Also ensembles with reduced input feature sets, using NPSFS, demonstrate as good or improved performance when compared with the full feature set models. Conclusions are drawn about a new set of tools and techniques, including NPSFS and visualisation techniques for inspection of ANN weights, the adoption of which it is hoped may lead to improved confidence in the use of ANN for live real-time EWS applications.

The primary goal of artificial intelligence research is to develop a machine capable of learning to solve disparate real-world tasks autonomously, without relying on specialized problem-specific inputs. This dissertation suggests that such machines are realistic: If No Free Lunch theorems were to apply to all real-world problems, then the world would be utterly unpredictable. In response, the dissertation proposes the information-maximization principle, which claims that the optimal optimization methods make the best use of the information available to them. This principle results in a new algorithm, evolutionary annealing, which is shown to perform well especially in challenging problems with irregular structure. / text

The theoretical part of this work deals with evolutionary algorithms (EA), neural networks (NN) and their synthesis in the form of neuroevolution. From a practical point of view, the aim of the work is to show the application of neuroevolution on two different tasks. The first task is the evolutionary design of the convolutional neural network (CNN) architecture that would be able to classify handwritten digits (from the MNIST dataset) with a high accurancy. The second task is the evolutionary optimization of neurocontroller for a simulated Falcon 9 rocket landing. Both tasks are computationally demanding and therefore have been solved on a supercomputer. As a part of the first task, it was possible to design such architectures which, when properly trained, achieve an accuracy of 99.49%. It turned out that it is possible to automate the design of high-quality architectures with the use of neuroevolution. Within the second task, the neuro-controller weights have been optimized so that, for defined initial conditions, the model of the Falcon booster can successfully land. Neuroevolution succeeded in both tasks.