Stop-and-Go Adaptive Cruise Control (ACC+) is an extension of Adaptive Cruise Control (ACC) that works at low speed as well as normal highway speeds to regulate the speed of the vehicle relative to the vehicle it is following. In this thesis, we design an ACC+ controller for a scale model electric vehicle that ensures the robust performance of the system under various models of uncertainty. We capture the operation of the hybrid system via a state-chart model that performs mode switching between different digital controllers with additional decision logic to guarantee the collision freedom of the system under normal operation. We apply different controller design methods such as Linear Quadratic Regulator (LQR) and H-infinity and perform multiple simulation runs in MATLAB/Simulink to validate the performance of the proposed designs. We compare the practicality of our design with existing formally verified ACC designs from the literature. The comparisons show that the other formally verified designs exhibit unacceptable behaviour in the form of mode thrashing that produces excessive acceleration and deceleration of the vehicle. While simulations provide some assurance of safe operation of the system design, they do not guarantee system safety under all possible cases. To increase confidence in the system, we use Differential Dynamic Logic (dL) to formally state environmental assumptions and prove safety goals, including collision freedom. The verification is done in two stages. First, we identify the invariant required to ensure the safe operation of the system and we formally verify that the invariant preserves the safety property of any system with similar dynamics. This procedure provides a high level abstraction of a class of safe solutions for ACC+ system designs. Second, we show that our ACC+ system design is a refinement of the abstract model. The safety of the closed loop ACC+ system is proven by verifying bounds on the system variables using the KeYmaera verification tool for hybrid systems. The thesis demonstrates how practical ACC+ controller designs optimized for fuel economy, passenger comfort, etc., can be verified by showing that they are a refinement of the abstract high level design. / Thesis / Master of Applied Science (MASc)

Energy storage systems with many Lithium Ion battery cells per string require sophisticated balancing hardware due to individual cells having manufacturing inconsistencies, different self discharge rates, internal resistances and temperature variations. For capacity maximization, safe operation, and extended lifetime, battery balancing is required. Redistributive Non-Dissipative balancing further improves the pack capacity and efficiency over a Dissipative approach where energy is wasted as heat across shunt resistors. Redistribution techniques dynamically shuttle charge to and from weak cells during operation such that all of the stored energy in the stack is utilized. This thesis identifies and develops different balancing control methods. These methods include a unconstrained optimization problem using a Linear Quadratic Regulator (LQR) and a constrained optimization problem using Model Predictive Control (MPC). These methods are benchmarked against traditional rule based (RB) balancing. The control systems are developed using MATLAB/Simulink and validated experimentally on a multiple transformer individual cell to stack topology. The implementation uses a DC2100A Demo-board from Linear Technology with bi-directional flyback converters to transfer the energy between the cells. The results of this thesis show that the MPC control method has the highest balancing efficiency and minimum balancing time. / Thesis / Master of Applied Science (MASc)

Orientador: Edvaldo Assunção / Resumo: Neste trabalho é proposta a resolução do problema do regulador linear quadrático (Linear Quadratic Regulator - LQR) via desigualdades matriciais lineares (Linear Matrix Inequalities - LMIs) para sistemas lineares e invariantes no tempo sujeitos a incertezas politópicas, bem como para sistemas lineares sujeitos a parâmetros variantes no tempo (Linear Parameter Varying - LPV). O projeto dos controladores é baseado na realimentação derivativa. A escolha da realimentação derivativa se dá devido à sua fácil implementação em certas aplicações como, por exemplo, no controle de vibrações. Os sinais usados na realimentação são aceleração e velocidade, sendo obtidos por meio de acelerômetros. Por meio do método proposto é possível obter condições LMIs para a síntese de controladores que garantam a estabilização do sistema em malha fechada, sendo que os controladores possuem desempenho otimizado. Para a formulação das condições LMIs, uma função de Lyapunov do tipo quadrática é utilizada. Exemplos teóricos e simulações são utilizados como forma de validação dos métodos propostos, além de mostrar que os novos resultados apresentam condições menos conservadoras. Além disso, ao final é apresentada uma implementação prática em um sistema de suspensão ativa, produzida pela Quanser®. / Abstract: The resolution of linear quadratic regulator (LQR) problem via linear matrix inequalities (LMIs) for linear time-invariant systems subject to polytopic uncertainties, as linear systems subjects to linear parameter varying (LPV), is proposed in this work. The controllers' designs are based on the state derivative feedback. The aim to the choice of the state derivative feedback is your easy implementation in a class of mechanical systems, such as in vibration control, for example. The signals used for feedback are acceleration and velocity, it is obtained by means of accelerometers. Through the proposed method it is possible to obtain LMIs conditions for the synthesis of controllers that guarantee the stabilisation of the closed-loop system, being that the controllers have optimised performance. For the LMIs conditions formulations, a Lyapunov function of type quadratic is used. As a form of validation, theoretical examples and simulations are performed, besides to show that the new results are less conservative. Furthermore, a practical implementation in an active suspension system, produced by Quanser®, is performed. / Mestre

Submitted by Rosivalda Pereira (mrs.pereira@ufma.br) on 2017-06-23T20:34:27Z No. of bitstreams: 1 ErnestoFerreira.pdf: 1744167 bytes, checksum: c125c90e5eb2aab2618350567f88cb31 (MD5) / Made available in DSpace on 2017-06-23T20:34:27Z (GMT). No. of bitstreams: 1 ErnestoFerreira.pdf: 1744167 bytes, checksum: c125c90e5eb2aab2618350567f88cb31 (MD5) Previous issue date: 2016-03-08 / The development and the numerical stability analysis of a new adaptive critic algorithm to approximate the state-value function for online discrete linear quadratic regulator (DLQR) optimal control system design based on heuristic dynamic programming (HDP) are presented in this work. The proposed algorithm makes use of unitary transformations and QR decomposition methods to improve the online learning e-ciency in the critic network through the recursive least-squares (RLS) approach. The developed learning strategy provides computational performance improvements in terms of numerical stability and computational cost which aim at making possible the implementations in real time of optimal control design methodology based upon actor-critic reinforcement learning paradigms. The convergence behavior and numerical stability of the proposed online algorithm, called RLSµ-QR-HDP-DLQR, are evaluated by computational simulations in three Multiple-Input and Multiple-Output (MIMO) models, that represent the automatic pilot of an F-16 aircraft of third order, a fourth order RLC circuit with two input voltages and two controllable voltage levels, and a doubly-fed induction generator with six inputs and six outputs for wind energy conversion systems. / Neste trabalho, apresenta-se o desenvolvimento e a análise da estabilidade numérica de um novo algoritmo crítico adaptativo para aproximar a função valor de estado para o projeto do sistema de controle ótimo online, utilizando o regulador linear quadrático discreto (DLQR), com base em programação dinâmica heurística (HDP). O algoritmo proposto faz uso de transformações unitárias e métodos de decomposição QR para melhorar a e-ciência da aprendizagem online na rede crítica por meio da abordagem dos mínimos quadrados recursivos (RLS). A estratégia de aprendizagem desenvolvida fornece melhorias no desempenho computacional em termos de estabilidade numérica e custo computacional, que visam tornar possíveis as implementações em tempo real da metodologia do projeto de controle ótimo com base em paradigmas de aprendizado por reforço ator-crítico. O comportamento de convergência e estabilidade numérica do algoritmo online proposto, denominado RLSµ-QR-HDP-DLQR, são avaliados por meio de simulações computacionais em três modelos Múltiplas-Entradas e Múltiplas-Saídas (MIMO), que representam o piloto automático de uma aeronave F-16 de terceira ordem, um circuito de quarta ordem RLC com duas tensões de entrada e dois níveis de tensão controláveis, e um gerador de indução duplamente alimentados com seis entradas e seis saídas para sistemas de conversão de energia eólica.

Submitted by Maria Aparecida (cidazen@gmail.com) on 2017-08-30T15:33:12Z No. of bitstreams: 1 Patricia Helena.pdf: 11110405 bytes, checksum: ca1f067231658f897d84b86181dbf1b9 (MD5) / Made available in DSpace on 2017-08-30T15:33:12Z (GMT). No. of bitstreams: 1 Patricia Helena.pdf: 11110405 bytes, checksum: ca1f067231658f897d84b86181dbf1b9 (MD5) Previous issue date: 2014-07-24 / In this thesis a proposal of an uni ed approach of dynamic programming, reinforcement learning and function approximation theories aiming at the development of methods and algorithms for design of optimal control systems is presented. This approach is presented in the approximate dynamic programming context that allows approximating the optimal feedback solution as to reduce the computational complexity associated to the conventional dynamic programming methods for optimal control of multivariable systems. Speci cally, in the state and action dependent heuristic dynamic programming framework, this proposal is oriented for the development of online approximated solutions, numerically stable, of the Riccati-type Hamilton-Jacobi-Bellman equation associated to the discrete linear quadratic regulator problem which is based on a formulation that combines value function estimates by means of a RLS (Recursive Least-Squares) structure, temporal di erences and policy improvements. The development of the proposed methodologies, in this work, is focused mainly on the UDU T factorization that is inserted in this framework to improve the RLS estimation process of optimal decision policies of the discrete linear quadratic regulator, by circumventing convergence and numerical stability problems related to the covariance matrix ill-conditioning of the RLS approach. / Apresenta-se nesta tese uma proposta de uma abordagem uni cada de teorias de programação dinâmica, aprendizagem por reforço e aproximação de função que tem por objetivo o desenvolvimento de métodos e algoritmos para projeto online de sistemas de controle ótimo. Esta abordagem é apresentada no contexto de programação dinâmica aproximada que permite aproximar a solução de realimentação ótima de modo a reduzir a complexidade computacional associada com métodos convencionais de programação dinâmica para controle ótimo de sistemas multivariáveis. Especi camente, no quadro de programação dinâmica heurística e programação dinâmica heurística dependente de ação, esta proposta é orientada para o desenvolvimento de soluções aproximadas online, numericamente estáveis, da equação de Hamilton-Jacobi-Bellman do tipo Riccati associada ao problema do regulador linear quadrático discreto que tem por base uma formulação que combina estimativas da função valor por meio de uma estrutura RLS (do inglês Recursive Least-Squares), diferenças temporais e melhorias de política. O desenvolvimento das metodologias propostas, neste trabalho, tem seu foco principal voltado para a fatoração UDU T que é inserida neste quadro para melhorar o processo de estimação RLS de políticas de decisão ótimas do regulador linear quadrá- tico discreto, contornando-se problemas de convergência e estabilidade numérica relacionados com o mal condicionamento da matriz de covariância da abordagem RLS.

Made available in DSpace on 2016-08-17T14:52:39Z (GMT). No. of bitstreams: 1 DISSERTACAO_DENIS FABRICIO SOUSA DE SA.pdf: 2409581 bytes, checksum: 4de5274676a1f75ffe2a1f6b46b1388c (MD5) Previous issue date: 2015-04-10 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / The representation of dynamic systems or plants via mathematical models occupies an important position in control system design that allow the performance evaluation of the controller during his development stage. These models are also used as an alternative to solve the problem of the hardness or impracticability to install sensors that measure the controlled variables, the dynamic systems representations enable non-invasive measurement of these variables. As consequence the designer has an alternative way to perform adaptive and optimal sensorless control for a given process. In this dissertation is presented a proposal for control systems schemas and algorithms, based on recurrent neural networks (ANN) and Box-Jenkins models, that are dedicated to sensorless or indirect control of dynamic systems. The proposed models and algorithms are associated with the systems identification and recurrent ANN approaches. The algorithms developed for the AAN training are Backpropagation Accelerated and RLS types that are compared with classical methods and strategies to obtain it online parameters of indirect control of system for a thermal plant, where the actuator is Peltier cell. The performance the parametric models of the plant and adaptive PID digital controllers and linear quadratic regulator (DLQR) that are the main elements of the sensorless temperature control system, are evaluated by means of hybrid simulations, where the algorithms implemented in micro controllers and the plant represented by mathematical models. The performance results of the proposed sensorless control algorithms are promissory, not only, in terms of the control system performance, but also due to the reexibility to deploy it in other dynamic systems. / A representação de sistemas dinâmicos ou plantas por meio modelos matemáticos ocupa uma posição relevante no projeto de sistemas de controle, permitindo que o projetista avalie o desempenho dos controladores durante a fase de desenvolvimento do projeto. Estes modelos também são utilizados para resolver o problema da dificuldade ou impossibilidade da inserção de sensores em plantas para medição de variáveis controladas, onde os modelos viabilizam a mediação não invasiva destas variáveis, fornecendo uma alternativa para realização do controle indireto adaptativo e ótimo de um dado processo. Nesta dissertação apresenta-se o desenvolvimento de modelos propostos baseados em redes neurais artificiais recorrentes para o controle sensorless ou indireto da planta. Os modelos propostos estão associados com as abordagens de Identificação de Sistemas e de RNA's recorrentes. OS algoritmos desenvolvidos para o treinamento das RNAs são do tipo Backpropagation acelerado e RLS, que são comparados com estratégias e métodos clássicos, para obtenção online dos parâmetros do sistema de controle indireto de uma planta térmica, tendo como atuador uma célula Peltier. Para uns de avaliação de desempenho do sistema de controle indireto da planta, os modelos paramétricos e controladores digitais adaptativos do tipo PID e regulador linear quadrático (DLQR) são avaliados por meio de simulações híbridas, sendo os algoritmos dos controladores implementados em microcontroladores e a planta representada por modelos matemáticos. Os resultados apresentados são promissores, não são sentido do desempenho do sistema de controle, mas também nos custos reduzidos para seu desenvolvimento, operação e flexibilidade de aplicação em outros sistemas dinâmicos.

Made available in DSpace on 2016-08-17T14:53:08Z (GMT). No. of bitstreams: 1 Marcio Mendes Cerqueira.pdf: 4042726 bytes, checksum: a65c6a7174271eecc553e3a5b0ceb33a (MD5) Previous issue date: 2010-02-05 / Conselho Nacional de Desenvolvimento Científico e Tecnológico / Motivated by the necessity of efficient algorithms, it s presented the development of a methodology for the design and analysis of state observers in open and closed loops that are dedicated to monitoring and control of dynamic systems. The development of observers are based on OE models, description in state space and Kalman filter. The models are evaluated for temperature control of a aluminum cube that is inside of a sterilizer oven. In addition to the models assessment in terms of its ability to represent behavior of plants, these models also evaluated for the design of discrete linear quadric regulator DLQR that are tuned by genetic algorithms. The monitoring models are evaluated for open and closed loops structures that are represented by algorithms in terms of difference equations, these algorithms are seen as software core for the indirect measurement systems. / Motivado pela necessidade de algoritmos eficientes, apresenta-se o desenvolvimento de uma metodologia para projeto e análise de observadores de estado em malhas aberta e fechada que são dedicados a monitoração e controle de sistemas dinâmicos. O desenvolvimento dos observadores estão fundamentados em modelos OE, descrição no espaço de estados e filtro de Kalman. Os modelos são avaliados para o controle da temperatura de um cubo de alumínio que encontra-se no interior de uma estufa. Além das avaliações dos modelos em termos de sua habilidade em representar comportamento de plantas, estes são também avaliados para o projeto do regulador linear quadrático discreto (RQLD) que são sintonizados por algoritmos genéticos. Aplicação dos modelos para monitoração é avaliada nas estruturas das malhas aberta e fechada que são representadas por algoritmos em da equação à diferença, tendo em vistas o desenvolvimento de núcleos de software para os sistemas de medição indireta.

Concorder un modèle à certaines observations, voilà qui résume assez bien ce que l’apprentissage machine cherche à accomplir. Ce concept est maintenant omniprésent dans nos vies, entre autre grâce aux percées récentes en apprentissage profond. La stratégie d’optimisation prédominante pour ces deux domaines est la minimisation d’un objectif donné. Et pour cela, la méthode du gradient, méthode de premier-ordre qui modifie les paramètres du modèle à chaque itération, est l’approche dominante. À l’opposé, les méthodes dites de second ordre n’ont jamais réussi à s’imposer en apprentissage profond. Pourtant, elles offrent des avantages reconnus qui soulèvent encore un grand intérêt. D’où l’importance de la méthode du col, qui unifie les méthodes de premier et second ordre sous un même paradigme. Dans ce mémoire, nous établissons un parralèle direct entre la méthode du col et le domaine du contrôle optimal ; domaine qui cherche à optimiser mathématiquement une séquence de décisions. Et certains des problèmes les mieux compris et étudiés en contrôle optimal sont les commandes linéaires quadratiques. Problèmes pour lesquels on connaît très bien la solution optimale. Plus spécifiquement, nous démontrerons l’équivalence entre une itération de la méthode du col et la résolution d’une Commande Linéaire Quadratique (CLQ). Cet éclairage nouveau implique une approche unifiée quand vient le temps de déployer nombre d’algorithmes issus de la méthode du col, tel que la méthode du gradient et celle des gradients naturels, sans être limitée à ceux-ci. Approche que nous étendons ensuite aux problèmes à horizon infini, tel que les modèles à équilibre profond. Ce faisant, nous démontrons pour ces problèmes que calculer les gradients via la différentiation implicite revient à employer l’équation de Riccati pour solutionner la CLQ associée à la méthode du gradient. Finalement, notons que l’incorporation d’information sur la courbure du problème revient généralement à rencontrer une inversion matricielle dans la méthode du col. Nous montrons que l’équivalence avec les CLQ permet de contourner cette inversion en utilisant une approximation issue des séries de Neumann. Surprenamment, certaines observations empiriques suggèrent que cette approximation aide aussi à stabiliser le processus d’optimisation quand des méthodes de second-ordre sont impliquées ; en agissant comme un régularisateur adaptif implicite. / Machine learning entails training a model to fit some given observations, and recent advances in the field, particularly in deep learning, have made it omnipresent in our lives. Fitting a model usually requires the minimization of a given objective. When it comes to deep learning, first-order methods like gradient descent have become a default tool for optimization in deep learning. On the other hand, second-order methods did not see widespread use in deep learning. Yet, they hold many promises and are still a very active field of research. An important perspective into both methods is steepest descent, which allows you to encompass first and second-order approaches into the same framework. In this thesis, we establish an explicit connection between steepest descent and optimal control, a field that tries to optimize sequential decision-making processes. Core to it is the family of problems known as Linear Quadratic Regulation; problems that have been well studied and for which we know optimal solutions. More specifically, we show that performing one iteration of steepest descent is equivalent to solving a Linear Quadratic Regulator (LQR). This perspective gives us a convenient and unified framework for deploying a wide range of steepest descent algorithms, such as gradient descent and natural gradient descent, but certainly not limited to. This framework can also be extended to problems with an infinite horizon, such as deep equilibrium models. Doing so reveals that retrieving the gradient via implicit differentiation is equivalent to recovering it via Riccati’s solution to the LQR associated with gradient descent. Finally, incorporating curvature information into steepest descent usually takes the form of a matrix inversion. However, casting a steepest descent step as a LQR also hints toward a trick that allows to sidestep this inversion, by leveraging Neumann’s series approximation. Empirical observations provide evidence that this approximation actually helps to stabilize the training process, by acting as an adaptive damping parameter.

Micro aerial vehicles and other autonomous systems have the potential to truly transform life as we know it, however much of the potential of autonomous systems remains unrealized because reliable navigation is still an unsolved problem with significant challenges. This dissertation presents solutions to many aspects of autonomous navigation. First, it presents ROSflight, a software and hardware architure that allows for rapid prototyping and experimentation of autonomy algorithms on MAVs with lightweight, efficient flight control. Next, this dissertation presents improvments to the state-of-the-art in optimal control of quadrotors by utilizing the error-state formulation frequently utilized in state estimation. It is shown that performing optimal control directly over the error-state results in a vastly more computationally efficient system than competing methods while also dealing with the non-vector rotation components of the state in a principled way. In addition, real-time robust flight planning is considered with a method to navigate cluttered, potentially unknown scenarios with real-time obstacle avoidance. Robust state estimation is a critical component to reliable operation, and this dissertation focuses on improving the robustness of visual-inertial state estimation in a filtering framework by extending the state-of-the-art to include better modeling and sensor fusion. Further, this dissertation takes concepts from the visual-inertial estimation community and applies it to tightly-coupled GNSS, visual-inertial state estimation. This method is shown to demonstrate significantly more reliable state estimation than visual-inertial or GNSS-inertial state estimation alone in a hardware experiment through a GNSS-GNSS denied transition flying under a building and back out into open sky. Finally, this dissertation explores a novel method to combine measurements from multiple agents into a coherent map. Traditional approaches to this problem attempt to solve for the position of multiple agents at specific times in their trajectories. This dissertation instead attempts to solve this problem in a relative context, resulting in a much more robust approach that is able to handle much greater intial error than traditional approaches.

This master thesis project is focused on the development of an LQR controller and its comparison with other controllers (PID and MPC), in order to successfully control an Autonomous Underwater Vehicle manipulation system. The modelling of the manipulator was performed first in Matlab and later on in Simulink-Simscape. Once the manipulator was integrated with the AUV model, the LQR controller was also developed initially in Matlab and then in Simulink. The controller was then extracted from Simulink as a C-code and verified in Stonefish. After confirming that the LQR code was working in Stonefish, its results from Simulink were compared with PID and MPC results for two different trajectories. The data for comparison and statistical analysis were divided into the two trajectory scenarios (horizontal and vertical) since the weight matrices of both controllers were different. Looking at the system’s overall behavior the Model Predictive Control (MPC) and LQR had similar results, regarding the rise time, overshoot, steady-state error and robustness to disturbances. An anticipated fact for the MPC was that it takes the longest run time for both scenarios. Lastly, as expected the PID had the worst response of all three controllers, in both scenarios. Implementing a PID on a nonlinear system, produced many oscillations without being able to stabilize at the reference value, thus giving a large steady-state error. In addition, it could not counteract the noise disturbances in the signal. / Detta examensarbete är inriktat på utvecklingen av en LQR-styrenhet och dess jämförelse med andra kontroller (PID och MPC), för att framgångsrikt styra ett autonomt undervattensfordon-manipulationssystem. Modelleringen av manipulatorn utfördes först i Matlab och senare i Simulink-Simscape. När manipulatorn väl hade integrerats med AUV modellen, utvecklades LQR styrenheten också inledningsvis i Matlab och sedan i Simulink. Kontrollenheten extraherades sedan från Simulink som en C-kod och verifierades i Stonefish. Efter att ha bekräftat att LQR koden fungerade i Stonefish, jämfördes resultaten från Simulink med PID och MPC resultat för två olika banor. Data för jämförelse och statistisk analys delades in i de två bana-scenarierna (horisontella och vertikala), eftersom viktmatriserna för båda kontrollerna var olika. När man tittar på systemets övergripande beteende hade Model Predictive Controller (MPC) och LQR liknande resultat när det gäller stigningstid, överskott, steady-state fel och robusthet mot störningar. Ett förväntat faktum för MPC var att det tar den längsta körtiden för båda scenarierna. Slutligen, som väntat, hade PID det sämsta svaret av alla tre kontrollerna, i båda scenarierna. Implementering av ett PID på ett olinjärt system gav många oscillationer utan att kunna stabilisera sig vid referensvärdet, vilket gav ett stort steady-state fel. Dessutom kunde den inte motverka bullerstörningarna i signalen.