The goal of Evolutionary Robotics (ER) is the development of automatic processes for the synthesis of robot control systems using evolutionary computation. The idea that it may be possible to synthesise robotic control systems using an automatic design process is appealing. However, ER is considerably more challenging and less automatic than its advocates would suggest. ER applies methods from the field of neuroevolution to evolve robot control systems. Neuroevolution is a machine learning algorithm that applies evolutionary computation to the design of Artificial Neural Networks (ANN). The aim of this thesis is to assay the practical characteristics of neuroevolution by performing bulk experiments on a set of Reinforcement Learning (RL) problems. This thesis was conducted with the view of applying neuroevolution to the design of neurocontrollers for small low-cost Autonomous Underwater Vehicles (AUV). A general approach to neuroevolution for RL problems is presented. The is selected to evolve ANN connection weights on the basis that it has shown competitive performance on continuous optimisation problems, is self-adaptive and can exploit dependencies between connection weights. Practical implementation issues are identified and discussed. A series of experiments are conducted on RL problems. These problems are representative of problems from the AUV domain, but manageable in terms of problem complexity and computational resources required. Results from these experiments are analysed to draw out practical characteristics of neuroevolution. Bulk experiments are conducted using the inverted pendulum problem. This popular control benchmark is inherently unstable, underactuated and non-linear: characteristics common to underwater vehicles. Two practical characteristics of neuroevolution are demonstrated: the importance of using randomly generated evaluation sets and the effect of evaluation noise on search performance. As part of these experiments, deficiencies in the benchmark are identified and modifications suggested. The problem of an underwater vehicle travelling to a goal in an obstacle free environment is studied. The vehicle is modelled as a Dubins car, which is a simplified model of the high-level kinematics of a torpedo class underwater vehicle. Two practical characteristics of neuroevolution are demonstrated: the importance of domain knowledge when formulating ANN inputs and how the fitness function defines the set of evolvable control policies. Paths generated by the evolved neurocontrollers are compared with known optimal solutions. A framework is presented to guide the practical application of neuroevolution to RL problems that covers a range of issues identified during the experiments conducted in this thesis. An assessment of neuroevolution concludes that it is far from automatic yet still has potential as a technique for solving reinforcement problems, although further research is required to better understand the process of evolutionary learning. The major contribution made by this thesis is a rigorous empirical study of the practical characteristics of neuroevolution as applied to RL problems. A critical, yet constructive, viewpoint is taken of neuroevolution. This viewpoint differs from much of the reseach undertaken in this field, which is often unjustifiably optimistic and tends to gloss over difficult practical issues.

O ambiente aquático tem notória importância para a pesquisa, pela biodiversidade e vastidão, e também do ponto de vista comercial, para a indústria militar e de óleo&gás por exemplo. Entretanto, a sua exploração é prejudicada por diversos fatores, entre eles devido à dificuldade de navegação. Infelizmente, carece-se de sinal GPS (Global Positioning System) embaixo d\'água, o que exige outras técnicas de localização. Assim, este trabalho analisa um sistema de navegação para um veículo submarino autônomo. Graças a sensores de velocidade, girômetros, bússola, entre outros, aplica-se o princípio de dead reckoning para calcular a posição atual do veículo a partir da última posição conhecida. Para tal, é feito inicialmente um estudo dos sensores a serem utilizados e um algoritmo de navegação é proposto, cujos resultados são expressos em coordenadas geodésicas (latitude e longitude), permitindo a visualização da trajetória do veículo em mapas geo-referenciados. Além disso, problemas práticos de medição são tratados. Em seguida, é feito um estudo sobre o ruído dos sensores, utilizando a curva de variância de Allan para caracterização dos sinais dos girômetros e do DVL (Doppler Velocity Logger). Por meio de equações de propagação de erro, os ruídos são recuperados em simulação, permitindo a estimação do erro de posição e de atitude (posição angular) acumulados para uma dada manobra. Finalmente, discute-se um critério de emersão a partir das estimativas de erro de posição. / The main part of our planet is filled with water, so the aquatic environment has notorious research and commercial importance. However, its exploration faces many difficulties. In navigation, the lack of GPS signal (Global Positioning System) during underwater missions requires different techniques, so this document focus on analyzing a navigation system for autonomous underwater vehicles. Thanks to different embedded sensors, like DVL (Doppler Velocity Logger), compass, gyrometers and others, the processes of dead reckoning is applied, witch calculates vehicle\'s current position by using the previously determined position. To do so, a navigation algorithm is implemented, providing geodesic coordinates to plot vehicle\'s trajectories on geo-referenced maps. Also, practical difficulties are discussed and treated. To improve the quality of the results, girometer\'s and DVL\'s errors are analyzed using Allan\'s variance and the navigation errors are estimated using first order time derivative equations in an augmented state space. Lastly, it is discussed a criterion to emerge and correct the vehicle\'s position using GPS signal.

Dissertação(mestrado) - Universidade Federal do Rio Grande, Programa de Pós-Graduação em Engenharia Oceânica, Escola de Engenharia, 2011. / Submitted by Lilian M. Silva (lilianmadeirasilva@hotmail.com) on 2013-04-22T19:33:16Z No. of bitstreams: 1 Modelagem e controle de atuadores robóticos e veículos subaquáticos não tripulados.pdf: 2108153 bytes, checksum: 01646a2e791884ffca750c5291ce8bd1 (MD5) / Approved for entry into archive by Bruna Vieira(bruninha_vieira@ibest.com.br) on 2013-06-03T19:14:00Z (GMT) No. of bitstreams: 1 Modelagem e controle de atuadores robóticos e veículos subaquáticos não tripulados.pdf: 2108153 bytes, checksum: 01646a2e791884ffca750c5291ce8bd1 (MD5) / Made available in DSpace on 2013-06-03T19:14:00Z (GMT). No. of bitstreams: 1 Modelagem e controle de atuadores robóticos e veículos subaquáticos não tripulados.pdf: 2108153 bytes, checksum: 01646a2e791884ffca750c5291ce8bd1 (MD5) Previous issue date: 2011 / Esta dissertação trata da modelagem e do controle de um atuador robótico e de veículos subaquáticos não tripulados. Primeiramente o trabalho compara e valida experimentalmente dois modelos de atrito existentes na literatura (LuGre e Gomes). Desenvolve-se um controle de posição baseado em estrutura variável para o atuador robótico do tipo harmonic-drive. Simulações são realizadas a fim de verificar a robustez do controlador perante a variação paramétrica em comparação com o controle proporcional, integral e derivativo (PID). Resultados experimentais das aplicações dos controles são obtidos mostrando desempenhos muito próximos aos de simulação. Posteriormente é feito um estudo sobre veículos subaquáticos não tripulados (Tatuí e Nerov). A modelagem cinemática é descrita por ângulos de Euler e a modelagem dinâmica inclui a interação da estrutura mecânica do veículo com o meio aquático e a ação dos propulsores. Desenvolve-se um sistema de controle de posição e orientação utilizando estrutura variável. Resultados de simulação ilustram as características do controlador com incertezas paramétricas e correntes oceânicas em comparação com o PID usando a estratégia de linearização por realimentação. / The subject of this dissertation is the modeling and control of a robotic actuator and also of unmanned underwater vehicles. Firstly, it compares and experimentally validates two friction models (LuGre and Gomes). A position control based on a variable structure is developed for the harmonic-drive type of robotic actuator. Simulations are performed to verify the robustness of the controller against the parametric variation and to compare to the proportional, integral and derivative control (PID). Experiments then show results similar to those from simulations. Later, we a study about unmanned underwater vehicles (Tatuí and Nerov) is presented. The kinematics modeling is described by Euler’s angles. The dynamics modeling includes the interaction of the vehicle’s mechanical structure with the aquatic environment and the actuation of the propellers. By using a variable structure, a position and orientation control system is developed. The simulation results then show the characteristics of the controler with parametric uncertainties and ocean streams, and compares them with those from a PID controler with feedback linearization.

Neste trabalho, estuda-se o sistema de controle distribuído de um veículo submarino não tripulado, assumindo-se um sistema com recursos computacionais limitados. A princípio, são discutidos as fontes e os efeitos dos atrasos de tempo introduzidos pela escassez de recursos. Após o detalhamento da modelagem do veículo LAURS, compara-se o desempenho de alguns compensadores de atraso encontrados na literatura. Para isso, um simulador do sistema de controle do veículo LAURS é desenvolvido com o auxílio de uma ferramenta para Matlab que permite simular os aspectos temporais de um sistema limitado. A partir dos resultados das simulações, incluindo-se o estudo de um cenário, conclue-se que embora os compensadores de atraso melhorem o desempenho do sistema de controle, o custo reduzido de um sistema computacional limitado não justifica a incerteza de desempenho que uma aplicação crítica, como é o caso do veículo submarino não tripulado, apresentará. O resultado das simulações é validado estatisticamente. / This work studies the networked control system of the unmanned underwater vehicle LAURS considering a resource-constrained computer system. At first, delays sources and effects are discussed. The LAURS model is obtained and used for comparison of delay compensation strategies found in the literature. A simulator for the LAURS control system is developed with the aid of a Matlab Toolbox that emulates timing aspects of the limited operating system and network. Considering the simulation results, including a case scenario, it is argued that despite improving performance, the control system with delay compensation strategies remains unpredictable and prone to fail. Therefore, it is not recommended for critical systems such as the unmanned vehicle LAURS, even with the reduced costs of a resource-constrained computer system. The simulation results are statistically validated.

Unmanned Underwater Vehicles (UUVs) are widely used in commercial, scientific, and military missions for various purposes. What makes this technology challenging is the increasing mission duration and unknown environment. It is necessary to embed fault-tolerant control paradigms into UUVs to increase the reliability of the vehicles and enable them to execute and finalize complex missions. Specifically, fault-tolerant control (FTC) comprises fault detection, identification, and control reconfiguration for fault compensation. Literature review shows that there have been no systematic methods for fault-tolerant control of UUVs in earlier investigations. This study presents a hierarchical methodology of fault detection, identification and compensation (HFDIC) that integrates these functions systematically in different levels. The method uses adaptive finite-impulse-response (FIR) modeling and analysis in its first level to detect failure occurrences. Specifically, it incorporates a FIR filter for on-line adaptive modeling, and a least-mean-squares (LMS) algorithm to minimize the output error between the monitored system and the filter in the modeling process. By analyzing the resulting adaptive filter coefficients, we extract the information on the fault occurrence. The HFDIC also includes a two-stage design of parallel Kalman filters in levels two and three for fault identification using the multiple-model adaptive estimation (MMAE). The algorithm activates latter levels only when the failure is detected, and can return back to the monitoring loop in case of false failures. On the basis of MMAE, we use multiple sliding-mode controllers and reconfigure the control law with a probability-weighted average of all the elemental control signals, in order to compensate for the fault. We validate the HFDIC on the steering and diving subsystems of Naval Postgraduate School (NPS) UUVs for various simulated actuator and/or sensor failures, and test the hierarchical fault detection and identification (HFDI) with realistic data from at-sea experiment of the Florida Atlantic University (FAU) Autonomous Underwater Vehicles (AUVs). For both occasions, we model actuator and sensor failures as additive parameter changes in the observation matrix and the output equation, respectively. Simulation results demonstrate the ability of the HFDIC to detect failures in real time, identify failures accurately with a low computational overhead, and compensate actuator and sensor failures with control reconfiguration. In particular, verification of HFDI with FAU data confirms the performance of the fault detection and identification methodology, and provides important information on the vehicle performance. / Ph. D.

Thesis (MScEng (Electrical and Electronic Engineering))--University of Stellenbosch, 2009. / ENGLISH ABSTRACT: Sonars are used extensively for underwater sensing and recent advances in forward-looking imaging sonar have made this type of sonar an appropriate choice for use on Autonomous Underwater Vehicles. The images received from these sonar do however, tend to be noisy and when used in shallow water contain strong bottom reflections that obscure returns from actual targets. The focus of this work was the investigation and development of post-processing techniques to enable the successful use of the sonar images for automated navigation. The use of standard image processing techniques for noise reduction and background estimation, were evaluated on sonar images with varying amounts of noise, as well as on a set of images taken from an AUV in a harbour. The use of multiple background removal and noise reduction techniques on a single image was also investigated. To this end a performance measure was developed, based on the dynamic range found in the image and the uniformity of returned targets. This provided a means to quantitatively compare sets of post-processing techniques and identify the “optimal” processing. The resultant images showed great improvement in the visibility of target areas and the proposed techniques can significantly improve the chances of correct target extraction. / AFRIKAANSE OPSOMMING: Sonars word algemeen gebruik as onderwater sensors. Onlangse ontwikkelings in vooruit-kykende sonars, maak hierdie tipe sonar ’n goeie keuse vir die gebruik op ’n Outomatiese Onderwater Voertuig. Die beelde wat ontvang word vanaf hierdie sonar neig om egter raserig te wees, en wanneer dit in vlak water gebruik word toon dit sterk bodemrefleksies, wat die weerkaatsings van regte teikens verduister. Die fokus van die werk was die ondersoek en ontwikkeling van naverwerkings tegnieke, wat die sonar beelde bruikbaar maak vir outomatiese navigasie. Die gebruik van standaard beeldverwerkingstegnieke vir ruis-onderdrukking en agtergrond beraming, is geëvalueer aan die hand van sonar beelde met verskillende hoeveelhede ruis, asook aan die hand van ’n stel beelde wat in ’n hawe geneem is. Verdere ondersoek is ingestel na die gebruik van meer as een agtergrond beramings en ruis onderdrukking tegniek op ’n enkele beeld. Hierdie het gelei tot die ontwikkeling van ’n maatstaf vir werkverrigting van toegepaste tegnieke. Hierdie maatstaf gee ’n kwantitatiewe waardering van die verbetering op die oorspronklike beeld, en is gebaseer op die verbetering in dinamiese bereik in die beeld en die uniformiteit van die teiken se weerkaatsing. Hierdie maatstaf is gebruik vir die vergelyking van verskeie tegnieke, en identifisering van die “optimale” verwerking. Die verwerkte beelde het ’n groot verbetering getoon in die sigbaarheid van teikens, en die voorgestelde tegnieke kan ’n betekenisvolle bedrae lewer tot die suksesvolle identifisering van obstruksies.

Approved for public release, distribution is unlimited / Autonomous Underwater Vehicles (AUVs) are now being introduced into the fleet to improve Mine Warfare capabilities. Several AUVs are under government-contracted development. Mission planning and data reporting vary between vehicles and systems. This variance does not pose an immediate problem, as only one AUV is currently in production. However, as more AUVs are put into production, commands will begin to get multiple AUVs. Without a single mission command language, multiple systems will require familiarity with multiple languages. Extensible Markup Language (XML) and related technologies may be used to facilitate interoperability between dissimilar AUVs and extract and integrate mission data into Navy C4I systems. XML makes archive maintenance easier, XML documents can be accessed via an http server, and, in root form, XML is transferable on the fly by stylesheet. This thesis presents an XML-based mission command for the command and control of AUVs. In addition, this thesis discusses XML technology and how XML is a viable means of achieving interoperability. Furthermore, this thesis provides an example mission file using existing software, and demonstrates the future of XML in AUV technology. Finally, this work ends with a compelling argument for the use of an XML-based mission command language to command all AUVs. / Ensign, United States Navy / Captain, United States Air Force

O presente trabalho apresenta a descrição de um modelo matemático completo em seis graus de liberdade para um Veículo Submersível Autônomo (VSA) sub-atuado. Desenvolveram-se métodos de identificação de sistemas para identificar o modelo não linear do veículo. A fim de evitar problemas de divergência na estimação de parâmetros hidrodinâmicos do modelo, usou-se o método de transformação paramétrica. Usou-se o filtro estendido de Kalman como estratégia para o processo de estimação de parâmetros quando ruídos de natureza gaussiana estavam presentes no modelo e nas medidas. Com o objetivo de estimar um maior número de parâmetros de uma só vez, empregou-se o método de máxima verossimilhança. Os experimentos mostraram que o filtro de Kalman responde bem à estimação de parâmetros específicos, porém, divergiu facilmente à estimação de múltiplos parâmetros. Uma alternativa que apresentou melhor desempenho foi o método de máxima verossimilhança. Testaram-se manobras circulares e de zig-zags para a obtenção de dados do veículo. Para os ensaios experimentais, utilizou-se o VSA sub-atuado do Laboratório de Veículos Não Tripulados (LVNT) do Departamento de Engenharia Mecatrônica da Escola Politécnica da Universidade de São Paulo. Validou-se o modelo identificado mediante o simulador do veículo. Numa segunda etapa, desenvolveram-se controladores H¥ capazes de controlar a dinâmica do VSA em seus seis graus de liberdade. Projetaram-se controladores SISO (uma entrada e uma saída) e MIMO (múltiplas entradas e múltiplas saídas) com o fim de avaliar o acoplamento dinâmico do sistema. Projetaram-se controladores centralizados robustos para garantir as condições de operação num ambiente marinho e em condições de laboratório próximas às de uma aplicação real. As leis de controle são baseadas na técnica de sensibilidade mista H¥ que garantem condições de robustez do sistema de controle, tanto no desempenho quanto na estabilidade. Uma estrutura de controle de dois graus de liberdade (2GL) produziu melhores propriedades de desempenho comparada com a estrutura do controlador de um grau de liberdade. Compararam-se as respostas dos controladores descentralizados SISO e os controladores centralizados. O controlador 2GL garantiu as especificações do projeto, inclusive aquelas definidas no domínio do tempo. Um controlador central pode controlar o veículo na realização de manobras complexas em três dimensões que emulem a inspeção ou monitoramento de sistemas offshores ou outras tarefas comuns na exploração submarinha. O trabalho apresenta também a integração dos algoritmos de controle com o sistema de tempo real embarcado, os sensores inerciais de navegação, os motores elétricos para os atuadores lemes e o propulsor, o banco de baterias e o processador central ARM7 de 32 bits de ponto fixo. Traduziram-se os algoritmos de controle de ordem elevada para a aritmética de ponto fixo produzindo a execução rápida e, no possível, evitando a ocorrência de transbordamento de dados. / This work presents a full six degrees-of-freedom mathematical model description of a subactuated Autonomous Underwater Vehicle (AUV). The work developed methods of System Identification for identifying the nonlinear model of the vehicle. In order to avoid divergence problems in the process of hydrodynamic, it used the parametric transformation technique. It used the extended Kalman filter to estimate the model parameters subject to Gaussian noise, in the process and in the measurements. In order to tackle the problem of multiple parameter estimation at once, the work used the maximum likelihood approach. The experimental results showed that the Kalman filter approach is better when the aim is to estimate a specific parameter, however, it diverges easily when the aim is to estimate multiple parameters. The maximum likelihood technique showed better response to estimate multiple parameters of the model. Zig-zag and circular standard maneuvers were tested with the identification algorithms. For experimental tests, an AUV, namely Pirajuba and constructed by the Unmanned Vehicle Laboratory (LVNT), were used. Results were also assessed using an AUV six degrees of freedom simulator. In a second stage, the work developed H¥ controllers to manoeuvre the vehicle in six-degrees-of-freedom. Decoupled SISO (single input and single output variables) and MIMO (multiple input and multiple output variables) controllers were synthesized in order to validate the coupling dynamics of the AUV. Moreover, centralized robust controllers were developed to control the vehicle in the sea and in test tanks with extreme conditions close to the ocean environmental. The control techniques were based in the H¥ mixed sensitivity approach which guarantees robust performance and stability of the sub-actuated system. A structure of two-degrees-of-freedom (2GL) controller presented better performance compared with the classic single H¥ controller of one degree of freedom structure. A comparison between responses was used to validate the decoupling and centralized controllers. The 2GL controller has good performance specifications despite these defined in the time domain. A central controller can control the AUV in complex maritime task that require complex and three-dimensional manoeuvres. The work deals also with the implementation issues coding these advanced control algorithms into the real time embedded system including inertial sensors, electric motors for the propeller and actuator surfaces, battery banks, and the unit central process ARM7 of 32 bits of fixed point. The control algorithms were translated from floating point to fixed point arithmetic avoiding data overflow, seeking simplicity and fast task execution.

Made available in DSpace on 2017-07-10T17:11:49Z (GMT). No. of bitstreams: 1 Antonio_Chiella.pdf: 1495489 bytes, checksum: c662fb0b19b070474b00c5dcbb37ba11 (MD5) Previous issue date: 2015-02-27 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / In recent years, there has been a growth in the interest of industry and the scientific community for autonomous platforms, among these so-called AUVs (Autonomous Underwater Vehicle). Part of this interest is due to technological progress, with improved electronics and the reduction of its cost. Another part is due to application of this equipment in a variety of tasks such as inspection and maintenance of underwater structures in marine environment (oil and gas platforms) and recently inspection of hydroelectric plants. These robots allow the removal the operator of the region conducting task, reducing the risks in carrying it. In an attempt to make these fully autonomous platforms, reducing the efforts of the operator, challenges related to its control position emerge. In this work, the AUVs positioning control problem is addressed. The kinematic and dynamic models for the 6 degrees of freedom of these robots, as well as the modeling of the actuators and the external disturbances are presented. Characteristics of the mathematical model are used in the controller design, titled CMDG (Global Sliding Mode Control). The proposed controller is based on the sliding mode control, and its sliding surface was modified so as to delete the so called reaching phase. Numerical simulations show the good performance of the proposed controller when subjected to disturbances such as current, non-zero buoyancy and initial position errors. / Nos últimos anos, houve um crescimento no interesse da indústria e da comunidade científica por plataformas autônomas, entre essas os chamados AUVs (do inglês, Autonomous Underwater Vehicle). Parte deste interesse se deve ao avanço tecnológico, com a melhoria dos equipamentos eletrônicos e a diminuição de seu custo. Outra parte se deve a aplicação destes equipamentos em tarefas variadas, como inspeção e manutenção de estruturas subaquáticas em ambiente marinho (plataformas de óleo e gás) e recentemente inspeção de usinas hidrelétricas. Estes robôs permitem o afastamento do operador da região de realização da tarefa, reduzindo os riscos na execução da mesma. Na tentativa de deixar estas plataformas totalmente autônomas, diminuindo os esforços do operador, desafios relacionados ao seu controle de posição emergem. Neste trabalho, o problema do controle de posicionamento de AUVs é abordado. O modelo cinemático e dinâmico para os 6 graus de liberdade destes robôs, bem como a modelagem dos atuadores e das perturbações externas são apresentados. Características do modelo matemático são utilizadas no projeto do controlador, intitulado de CMDG (Controle por Modos Deslizantes Global). O controlador proposto, é baseado no controle por modos deslizantes, sendo sua superfície de deslizamento modificada, de forma a se eliminar a chamada fase de alcance. Simulações numéricas, mostram o bom desempenho do controlador proposto quando submetido a perturbações como correnteza, flutuabilidade não nula e erros de posição inicial.

A supercavitating vehicle, a next-generation underwater vehicle capable of changing the paradigm of modern marine warfare, exploits supercavitation as a means to reduce drag and achieve extremely high submerged speeds. In supercavitating flows, a low-density gaseous cavity entirely envelops the vehicle and as a result the vehicle is in contact with liquid water only at its nose and partially over the afterbody. Hence, the vehicle experiences a substantially reduced skin drag and can achieve much higher speed than conventional vehicles. The development of a controllable and maneuvering supercavitating vehicle has been confronted with various challenging problems such as the potential instability of the vehicle, the unsteady nature of cavity dynamics, the complex and non-linear nature of the interaction between vehicle and cavity. Furthermore, major questions still need to be resolved regarding the basic configuration of the vehicle itself, including its control surfaces, the control system, and the cavity dynamics. In order to answer these fundamental questions, together with many similar ones, this dissertation develops an integrated simulation-based design tool to optimize the vehicle configuration subjected to operational design requirements, while predicting the complex coupled behavior of the vehicle for each design configuration. Particularly, this research attempts to include maneuvering flight as well as various operating trim conditions directly in the vehicle configurational optimization. This integrated approach provides significant improvement in performance in the preliminary design phase and indicates that trade-offs between various performance indexes are required due to their conflicting requirements. This dissertation also investigates trim conditions and dynamic characteristics of supercavitating vehicles through a full 6 DOF model. The influence of operating conditions, and cavity models and their memory effects on trim is analyzed and discussed. Unique characteristics are identified, e.g. the cavity memory effects introduce a favorable stabilizing effect by providing restoring fins and planing forces. Furthermore, this research investigates the flight envelope of a supercavitating vehicle, which is significantly different from that of a conventional vehicle due to different hydrodynamic coefficients as well as unique operational conditions.