In this paper, the use of reinforcement learning (RL) in control systems is investigated using a rotatory inverted pendulum as an example. The control behavior of an RL controller is compared to that of traditional LQR and MPC controllers. This is done by evaluating their behavior under optimal conditions, their disturbance behavior, their robustness and their development process. All the investigated controllers are developed using MATLAB and the Simulink simulation environment and later deployed to a real pendulum model powered by a Raspberry Pi. The RL algorithm used is Proximal Policy Optimization (PPO). The LQR controller exhibits an easy development process, an average to good control behavior and average to good robustness. A linear MPC controller could show excellent results under optimal operating conditions. However, when subjected to disturbances or deviations from the equilibrium point, it showed poor performance and sometimes instable behavior. Employing a nonlinear MPC Controller in real time was not possible due to the high computational effort involved. The RL controller exhibits by far the most versatile and robust control behavior. When operated in the simulation environment, it achieved a high control accuracy. When employed in the real system, however, it only shows average accuracy and a significantly greater performance loss compared to the simulation than the traditional controllers. With MATLAB, it is not yet possible to directly post-train the RL controller on the Raspberry Pi, which is an obstacle to the practical application of RL in a prototyping or teaching setting. Nevertheless, RL in general proves to be a flexible and powerful control method, which is well suited for complex or nonlinear systems where traditional controllers struggle.

No description available.

Optimal kontroll är studien om hur man på bästa sätt kan kontrollera ett system, givet kontrollvariabler, för att minimera ett specifikt prestationsindex. Denna teori användes för att studera hur en cykel optimalt skulle kunna balanseras. Cykeln modellerades med tre tillståndsparametrar, dess lutning theta, vinkelhastigheten hos lutningen theta' samt dess styrvinkeln alpha, som togs fram ifrån två differentialekvationer. Dessa tillstånd, tillsammans med en kontrollvariabel delta=alpha' (styrets vinkelhastighet), användes sedan med dynamisk programmering samt LQR-kontroll, två grenar inom optimal kontroll, för att undersöka kontroll av cykeln. Resultaten visar på att alla tillståndsparametrar konvergerar mot noll för båda metoderna innan sluttiden vilket indikerar på att cykeln lyckades balanseras, i teorin. Kontrollen som användes, styrets vinkelhastighet, var även rimlig och uppfattades som intuitiva.

This work focuses on a comparison of numerical methods for linear quadratic regulator (LQR) problems defined by parabolic partial differential equations. In particular, we study various methods for computing functional gains to boundary control problems for the heat equation. These methods require us to solve various equations including the algebraic Riccati equation, the Riccati partial differential equation and the Chandrasekhar partial differential equations. Numerical results are presented for control of a one-dimensional and a two-dimensional heat equation with Dirichlet or Robin boundary control. / Ph. D.

Atualmente, existe uma grande variedade de metodologias de controle que poderiam ser utilizadas e aplicadas a processos industriais. Algumas destas metodologias têm um projeto complexo, exigindo um estudo extra de engenharia para projetar um controlador com um desempenho excelente. Outros controladores não apresentam um desempenho tão bom, embora apresentem um projeto extremamente simples. Entre a mais simples e a mais complexa metodologia, existem soluções de controle intermediárias, baseadas nas teorias de controle ótimo, que apresentam uma relativa simplicidade de projeto combinada com um desempenho superior. A presente tese apresenta a aplicação de duas técnicas de controle aplicadas a uma Rede de Trocadores de Calor (RTC) com bypasses: o controle LQR (Regulador Linear Quadrático) e o controle H-Infinito, apresentando os resultados obtidos em simulação no Matlab/Simulink e também resultados experimentais. Além disso, foi desenvolvido um procedimento para a validação experimental de um modelo matemático do trocador de calor casco e tubo 1-1 proposto por NOVAZZI (2006). A obtenção dos resultados experimentais foi realizada através de uma RTC em escala piloto, projetada e construída no Laboratório de Engenharia Química do Centro Universitário da FEI, com a instrumentação necessária e devidamente interconectada a um computador PC através de uma placa de aquisição de dados. A simulação e os ensaios experimentais realizados com os controladores LQR e H-Infinito aplicados na RTC apresentaram resultados adequados quando comparados à dinâmica da RTC em malha aberta (600s.), pois exigiram em torno de 200s. para impor uma mudança de setpoint ou rejeitar os distúrbios aplicados na vazão fria e na temperatura quente de entrada. Além disso, os controladores foram capazes de eliminar as interações entre as variáveis de processo. / Nowadays, there are a lot of different control methodologies that could be used on industrial processes. Some of these methodologies have a complex design and also demands an extra engineering effort to design the controller with a superior performance. Some other controllers may not lead to a desirable performance although they are too easy to design. Midway between the simplest and the most complex controller designs, there are an intermediate solutions based on optimal control theory, that present a relative simplicity in design combined with a superior performance. The contribution of this work is to apply the LQR control (Linear Quadratic Regulator) and the H-Infinity control on a Heat Exchanger Network (HEN) with bypasses, presenting the simulations and the experimental results. An experimental validation of the shell and tube heat exchanger mathematical model was successfully developed based on a procedure proposed. The experimental results were obtained with an HEN assembled on the Chemical Engineering Laboratory located at Centro Universitario da FEI. The instrumentation was adequately performed and the signals were interconnected on PC computer by using a data acquisition card. The simulation and the experimental results with the LQR and the H-Infinity control demanded 200s. to achieve a new steady state hen disturbances or set point variation were applied on the HEN. Compared with the HEN setting time (600s.), the controllers demonstrated reasonable results to perform a disturbance rejection and a set point variation.

Le cancer est une maladie caractérisée par la croissance incontrôlée des cellules. Le nombre de personnes atteintes par le carcinome hépatocellulaire (CHC) est en progression croissante. Les traitements utilisés jusqu'à présent par les médecins tels que la chimioembolisation transartérielle (TACE) et la radioembolisation transartérielle (TARE) présentent des limitations à cause des effets secondaires causés sur les tissus sains. En vue d'atteindre un meilleur contrôle tumoral avec le minimum de complications des tissus sains, les approches microrobotiques peuvent apporter des solutions au problème du ciblage thérapeutique. Une solution consiste à contrôler la direction de transporteurs thérapeutiques (bolus magnétiques), composés de microparticules magnétiques et d’agents anti-cancéreux, à l’intérieur des vaisseaux sanguins vers la zone tumorale. Des champs magnétiques extérieurs sont alors utilisés pour propulser, guider et naviguer une flottille de bolus magnétiques au travers du réseau artériel. Cette thèse propose donc une méthodologie globale à mettre en place afin de rendre les procédures locorégionales transartérielles robotisées plus ciblées et plus localisées. Dans un premier temps, nous avons optimisé la quantité de médicament à injecter sous forme de bolus magnétiques. Ensuite, nous nous sommes intéressés à l'optimisation de la structure du bolus en vue d’assurer d’une part, la navigation optimale à l’intérieur des vaisseaux et d’autre part, d’offrir la possibilité d’embarquer une quantité d’agents thérapeutiques plus importante. La navigation des bolus délivrés par un cathéter vers la zone ciblée (tumeur) est assurée grâce au développement et à l'implémentation d’une loi de commande optimale. La validation de l'injection et de la navigation des bolus magnétiques a été réalisée sur une plateforme magnétique robotisée développée dans le cadre de cette thèse. / Cancer is a disease characterized by an uncontrolled cell growth. The number of people with hepatocellular carcinoma (HCC) is growing constantly. The treatments used by doctors until nowadays such as transarterial chemoembolization (TACE) and transarterial Radioembolization (TARE) have limitations because of the side effects caused to healthy tissues. In order to achieve best tumor control with minimal complications on healthy tissues, microrobotics technology can provide solutions to the problem of therapeutic targeting. One solution is to control the direction of the therapeutic carriers (magnetic bolus), composed of magnetic microparticles and anti-cancer agents, inside the blood vessels to the tumor area (target). External magnetic fields are then used to propel, steer and navigate a magnetic bolus fleet through the arterial system. This thesis offers a global methodology to implement in order to make the robotic transarterial locoregional procedures more targeted and localized. First, we have optimized the amount of drug to be injected as magnetic boluses. Then, we have carried out the optimization of the magnetic bolus structure in order to ensure firstly, the optimal navigation inside the vessels and secondly, to offer the possibility of carrying a larger amount of therapeutic agents. The navigation of boluses delivered by the catheter to the target area (tumor) is ensured through the development and implementation of the optimal control law. The validation of the injection and navigation magnetic bolus are performed on a magnetic microrobotic platform.

Atualmente, existe uma grande variedade de metodologias de controle que poderiam ser utilizadas e aplicadas a processos industriais. Algumas destas metodologias têm um projeto complexo, exigindo um estudo extra de engenharia para projetar um controlador com um desempenho excelente. Outros controladores não apresentam um desempenho tão bom, embora apresentem um projeto extremamente simples. Entre a mais simples e a mais complexa metodologia, existem soluções de controle intermediárias, baseadas nas teorias de controle ótimo, que apresentam uma relativa simplicidade de projeto combinada com um desempenho superior. A presente tese apresenta a aplicação de duas técnicas de controle aplicadas a uma Rede de Trocadores de Calor (RTC) com bypasses: o controle LQR (Regulador Linear Quadrático) e o controle H-Infinito, apresentando os resultados obtidos em simulação no Matlab/Simulink e também resultados experimentais. Além disso, foi desenvolvido um procedimento para a validação experimental de um modelo matemático do trocador de calor casco e tubo 1-1 proposto por NOVAZZI (2006). A obtenção dos resultados experimentais foi realizada através de uma RTC em escala piloto, projetada e construída no Laboratório de Engenharia Química do Centro Universitário da FEI, com a instrumentação necessária e devidamente interconectada a um computador PC através de uma placa de aquisição de dados. A simulação e os ensaios experimentais realizados com os controladores LQR e H-Infinito aplicados na RTC apresentaram resultados adequados quando comparados à dinâmica da RTC em malha aberta (600s.), pois exigiram em torno de 200s. para impor uma mudança de setpoint ou rejeitar os distúrbios aplicados na vazão fria e na temperatura quente de entrada. Além disso, os controladores foram capazes de eliminar as interações entre as variáveis de processo. / Nowadays, there are a lot of different control methodologies that could be used on industrial processes. Some of these methodologies have a complex design and also demands an extra engineering effort to design the controller with a superior performance. Some other controllers may not lead to a desirable performance although they are too easy to design. Midway between the simplest and the most complex controller designs, there are an intermediate solutions based on optimal control theory, that present a relative simplicity in design combined with a superior performance. The contribution of this work is to apply the LQR control (Linear Quadratic Regulator) and the H-Infinity control on a Heat Exchanger Network (HEN) with bypasses, presenting the simulations and the experimental results. An experimental validation of the shell and tube heat exchanger mathematical model was successfully developed based on a procedure proposed. The experimental results were obtained with an HEN assembled on the Chemical Engineering Laboratory located at Centro Universitario da FEI. The instrumentation was adequately performed and the signals were interconnected on PC computer by using a data acquisition card. The simulation and the experimental results with the LQR and the H-Infinity control demanded 200s. to achieve a new steady state hen disturbances or set point variation were applied on the HEN. Compared with the HEN setting time (600s.), the controllers demonstrated reasonable results to perform a disturbance rejection and a set point variation.

Universidade Federal do Cearà / A teoria de controle robusto evoluiu consideravelmente ao longo das Ãltimas dÃcadas, apresentando soluÃÃes para vÃrios tipos de problemas de anÃlise, desempenho e sÃntese de sistemas lineares incertos. As desigualdades matriciais lineares (LMIs) e suas tÃcnicas surgiram como poderosas ferramentas em diversas Ãreas de engenharia de controle para projetos estruturais. Uma propriedade importante das LMIs reside no fato de que o seu conjunto soluÃÃo à convexo. Esta propriedade à fundamental para que se possam formular problemas em controle robusto como sendo problemas de otimizaÃÃo convexa que minimizam uma funÃÃo objetivo. Diante destas afirmaÃÃes o presente trabalho utiliza um sistema de nÃvel de lÃquido com dois tanques interligados como planta onde a mesma foi modelada, e, em seguida, foi desenvolvido um controlador para garantir a sua estabilidade quadrÃtica, quando submetido a perturbaÃÃes externas incertas definidas em um politopo. Utilizou-se o regulador linear quadrÃtico com aÃÃo integral (LQI) como controlador, porÃm, o conceito Ãtimo do LQR nÃo leva em consideraÃÃo as incertezas paramÃtricas existentes nas plantas de projeto, com isso, foi apresentado um mÃtodo de resoluÃÃo do LQR utilizando otimizaÃÃo convexa. O LQR otimizado via LMIs permite a adiÃÃo de incertezas para a obtenÃÃo do ganho de realimentaÃÃo de estado. Os resultados obtidos comprovaram que a estratÃgia de controle LQI via resoluÃÃo LMI à eficaz como controle robusto, pois à capaz de incluir caracterÃsticas referentes à imprecisÃo do processo, alÃm disso, o controle LQI garante a otimalidade do controle. / The robust control theory has evolved considerably over the past decades, providing solutions for various problems of analysis, synthesis and performance of uncertain linear systems. The linear matrix inequalities (LMI) and its techniques have emerged as powerful tools in various areas of control engineering for structural projects. An important property of LMIs is the fact that its solution set is convex. This property is crucial in order to be able to make robust control problems as convex optimization problems that minimize an objective function. Given these statements the present work uses a liquid level system with two tanks connected to the plant where it was modeled, and then a controller is designed to ensure quadratic stability when subjected to external disturbances defined in an uncertain polytope. We used the linear quadratic regulator with integral action (LQI) as a controller, however, the concept of optimal LQR does not take into account the parametric uncertainties in the existing plant design, with it, was presented a method of solving the LQR using convex optimization. LQR optimized via LMI allows the addition of uncertainty to obtain the state feedback gain. The results obtained proved that the strategy of LQI control via LMI resolution is effective as robust control, because it can include features related to the imprecision of the process, moreover, the LQI control ensures the optimality of control.

Submitted by Renata Lopes (renatasil82@gmail.com) on 2017-04-20T17:37:36Z No. of bitstreams: 1 anasophiacavalcantialves.pdf: 26969611 bytes, checksum: 43d8f3d71a0dbcb949394e62ac48bbf4 (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2017-04-24T16:49:48Z (GMT) No. of bitstreams: 1 anasophiacavalcantialves.pdf: 26969611 bytes, checksum: 43d8f3d71a0dbcb949394e62ac48bbf4 (MD5) / Made available in DSpace on 2017-04-24T16:49:48Z (GMT). No. of bitstreams: 1 anasophiacavalcantialves.pdf: 26969611 bytes, checksum: 43d8f3d71a0dbcb949394e62ac48bbf4 (MD5) Previous issue date: 2012-11-27 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / O presente trabalho tem como objetivo aplicar e comparar técnicas de controle lineares e não lineares no controle de estabilidade de um quadricóptero. Inicialmente, apresenta-se um modelo dinâmico da aeronave para a simulação e controle do sistema. Em seguida, descreve-se os princípios de funcionamento do veículo e algumas características importantes na construção do quadricóptero. Com base no modelo dinâmico, as técnicas de controle são utilizadas para projetar os controladores. Quatro diferentes controladores foram projetados: um controlador PID, um controlador LQR, um controlador com base na teoria de Lyapunov e um controlador utilizando a técnica de Backstepping. Eles são aplicados para controlar a atitude da aeronave, tendo como principal tarefa a estabilização do quadricóptero em voo. Por fim, os controladores são comparados, validados e os resultados das simulações e da implementação real no quadricóptero são apresentados. Os melhores resultados para o controle de estabilidade do quadricóptero são obtidos utilizando a técnica de controle não linear Backstepping. / This present work aims to apply and compare linear and nonlinear control techniques in attitude stabilization of a quadricopter. At first, is presented a aircraft dynamic model for simulation and control of the system. Then, will be described the principles of operation and some important characteristics about quadricopter’s assembling. Based on dynamic model, the control techniques are used to design the controllers. Four different controllers were designed: a PID controller, a LQR controller, a Lyapunov’s stability theory based controller, and a backstepping controller. They are applied to control the aircraft attitude, having as main task the inflight stabilization of quadricopter. At the end, the controllers are compared, validated and the simulation results and real implementation are presented. The best results of quadricopter attitude control are obtained using the nonlinear control technique of Backstepping.

The diploma thesis deals with control of rotary inverted pendulum - Furuta pendulum. Solution for power electronics, sensors and coupling with PC is described, identification of parameters and nonlinear simulation model in Matlab/Simulink and SimMechanics toolbox is presented. Second type Lagrange equation is used for determination of equations of motion. Controll system based on state-space model of mechanism and LQR algorithm for design of state-space controller is used and switching between swing-up cotroller a stabilizing state-space control is achieved according to actual angular position of pendulum's angle. Input integrator eliminating steady state error was used with success.