Un modèle de croissance et de réponse à la radiothérapie pour le glioblastome multiforme (GBM) basé le formalisme du modèle de prolifération-invasion (PI) et du modèle linéaire-quadratique a été développé et implémenté. La géométrie spécifique au patient est considérée en modélisant, d'une part, les voies d'invasion possibles des GBM avec l'imagerie du tenseur de diffusion (DTI) et, d'autre part, les barrières à la propagation à partir des images anatomiques disponibles. La distribution de dose réelle reçue par un patient donné est appliquée telle quelle dans les simulations, en respectant l'horaire de traitement. Les paramètres libres du modèle (taux de prolifération, coefficient de diffusion, paramètres radiobiologiques) sont choisis aléatoirement à partir de distributions de valeurs plausibles. Un total de 400 ensembles de valeurs pour les paramètres libres sont ainsi choisis pour tous les patients, et une simulation de la croissance et de la réponse au traitement est effectuée pour chaque patient et chaque ensemble de paramètres. Un critère de récidive est appliqué sur les résultats de chaque simulation pour identifier un lieu probable de récidive (SPR). La superposition de tous les SPR obtenus pour un patient donné permet de définir la probabilité d'occurrence (OP). Il est démontré qu'il existe des valeurs de OP élevées pour tous les patients, impliquant que les résultats du modèle PI ne sont pas très sensibles aux valeurs des paramètres utilisés. Il est également démontré comment le formalisme développé dans cet ouvrage pourrait permettre de définir un volume cible personnalisé pour les traitements de radiothérapie du GBM. / We have developed and implemented a model of growth and response to radiotherapy for glioblastoma multiforme (GBM) based on the proliferation-invasion (PI) formalism and linear-quadratic model. We take into account patient-specific geometry to model the possible invasion pathways of GBM with diffusion tensor imaging (DTI) and the barriers to dispersal from anatomical images available. The actual dose distribution received by a given patient is applied as such in the simulation, respecting the treatment schedule. The free parameters in the model (proliferation rate, diffusion coefficient, radiobiological parameters) are randomly chosen from a distribution of plausible values. A total of 400 sets of values for the free parameters are thus chosen for all patients, and a simulation of the growth and the response to treatment is performed for each patient and each set of parameters. A failure criterion is applied to the results of each simulation to identify a site of potential recurrence (SPR). The superposition of all SPR obtained for a given patient defines the occurrence probability (OP). We show that high OP values exist for all patients and conclude that the PI model results are not very sensitive to the values of the parameters used. Finally, we show how the formalism developed in this work could help to define a custom target volume for radiation treatment of GBM.

Orientador: Edvaldo Assunção / Resumo: Neste trabalho é proposta a resolução do problema do regulador linear quadrático (Linear Quadratic Regulator - LQR) via desigualdades matriciais lineares (Linear Matrix Inequalities - LMIs) para sistemas lineares e invariantes no tempo sujeitos a incertezas politópicas, bem como para sistemas lineares sujeitos a parâmetros variantes no tempo (Linear Parameter Varying - LPV). O projeto dos controladores é baseado na realimentação derivativa. A escolha da realimentação derivativa se dá devido à sua fácil implementação em certas aplicações como, por exemplo, no controle de vibrações. Os sinais usados na realimentação são aceleração e velocidade, sendo obtidos por meio de acelerômetros. Por meio do método proposto é possível obter condições LMIs para a síntese de controladores que garantam a estabilização do sistema em malha fechada, sendo que os controladores possuem desempenho otimizado. Para a formulação das condições LMIs, uma função de Lyapunov do tipo quadrática é utilizada. Exemplos teóricos e simulações são utilizados como forma de validação dos métodos propostos, além de mostrar que os novos resultados apresentam condições menos conservadoras. Além disso, ao final é apresentada uma implementação prática em um sistema de suspensão ativa, produzida pela Quanser®. / Abstract: The resolution of linear quadratic regulator (LQR) problem via linear matrix inequalities (LMIs) for linear time-invariant systems subject to polytopic uncertainties, as linear systems subjects to linear parameter varying (LPV), is proposed in this work. The controllers' designs are based on the state derivative feedback. The aim to the choice of the state derivative feedback is your easy implementation in a class of mechanical systems, such as in vibration control, for example. The signals used for feedback are acceleration and velocity, it is obtained by means of accelerometers. Through the proposed method it is possible to obtain LMIs conditions for the synthesis of controllers that guarantee the stabilisation of the closed-loop system, being that the controllers have optimised performance. For the LMIs conditions formulations, a Lyapunov function of type quadratic is used. As a form of validation, theoretical examples and simulations are performed, besides to show that the new results are less conservative. Furthermore, a practical implementation in an active suspension system, produced by Quanser®, is performed. / Mestre

Submitted by Rosivalda Pereira (mrs.pereira@ufma.br) on 2017-06-23T20:34:27Z No. of bitstreams: 1 ErnestoFerreira.pdf: 1744167 bytes, checksum: c125c90e5eb2aab2618350567f88cb31 (MD5) / Made available in DSpace on 2017-06-23T20:34:27Z (GMT). No. of bitstreams: 1 ErnestoFerreira.pdf: 1744167 bytes, checksum: c125c90e5eb2aab2618350567f88cb31 (MD5) Previous issue date: 2016-03-08 / The development and the numerical stability analysis of a new adaptive critic algorithm to approximate the state-value function for online discrete linear quadratic regulator (DLQR) optimal control system design based on heuristic dynamic programming (HDP) are presented in this work. The proposed algorithm makes use of unitary transformations and QR decomposition methods to improve the online learning e-ciency in the critic network through the recursive least-squares (RLS) approach. The developed learning strategy provides computational performance improvements in terms of numerical stability and computational cost which aim at making possible the implementations in real time of optimal control design methodology based upon actor-critic reinforcement learning paradigms. The convergence behavior and numerical stability of the proposed online algorithm, called RLSµ-QR-HDP-DLQR, are evaluated by computational simulations in three Multiple-Input and Multiple-Output (MIMO) models, that represent the automatic pilot of an F-16 aircraft of third order, a fourth order RLC circuit with two input voltages and two controllable voltage levels, and a doubly-fed induction generator with six inputs and six outputs for wind energy conversion systems. / Neste trabalho, apresenta-se o desenvolvimento e a análise da estabilidade numérica de um novo algoritmo crítico adaptativo para aproximar a função valor de estado para o projeto do sistema de controle ótimo online, utilizando o regulador linear quadrático discreto (DLQR), com base em programação dinâmica heurística (HDP). O algoritmo proposto faz uso de transformações unitárias e métodos de decomposição QR para melhorar a e-ciência da aprendizagem online na rede crítica por meio da abordagem dos mínimos quadrados recursivos (RLS). A estratégia de aprendizagem desenvolvida fornece melhorias no desempenho computacional em termos de estabilidade numérica e custo computacional, que visam tornar possíveis as implementações em tempo real da metodologia do projeto de controle ótimo com base em paradigmas de aprendizado por reforço ator-crítico. O comportamento de convergência e estabilidade numérica do algoritmo online proposto, denominado RLSµ-QR-HDP-DLQR, são avaliados por meio de simulações computacionais em três modelos Múltiplas-Entradas e Múltiplas-Saídas (MIMO), que representam o piloto automático de uma aeronave F-16 de terceira ordem, um circuito de quarta ordem RLC com duas tensões de entrada e dois níveis de tensão controláveis, e um gerador de indução duplamente alimentados com seis entradas e seis saídas para sistemas de conversão de energia eólica.

Submitted by Maria Aparecida (cidazen@gmail.com) on 2017-08-30T15:33:12Z No. of bitstreams: 1 Patricia Helena.pdf: 11110405 bytes, checksum: ca1f067231658f897d84b86181dbf1b9 (MD5) / Made available in DSpace on 2017-08-30T15:33:12Z (GMT). No. of bitstreams: 1 Patricia Helena.pdf: 11110405 bytes, checksum: ca1f067231658f897d84b86181dbf1b9 (MD5) Previous issue date: 2014-07-24 / In this thesis a proposal of an uni ed approach of dynamic programming, reinforcement learning and function approximation theories aiming at the development of methods and algorithms for design of optimal control systems is presented. This approach is presented in the approximate dynamic programming context that allows approximating the optimal feedback solution as to reduce the computational complexity associated to the conventional dynamic programming methods for optimal control of multivariable systems. Speci cally, in the state and action dependent heuristic dynamic programming framework, this proposal is oriented for the development of online approximated solutions, numerically stable, of the Riccati-type Hamilton-Jacobi-Bellman equation associated to the discrete linear quadratic regulator problem which is based on a formulation that combines value function estimates by means of a RLS (Recursive Least-Squares) structure, temporal di erences and policy improvements. The development of the proposed methodologies, in this work, is focused mainly on the UDU T factorization that is inserted in this framework to improve the RLS estimation process of optimal decision policies of the discrete linear quadratic regulator, by circumventing convergence and numerical stability problems related to the covariance matrix ill-conditioning of the RLS approach. / Apresenta-se nesta tese uma proposta de uma abordagem uni cada de teorias de programação dinâmica, aprendizagem por reforço e aproximação de função que tem por objetivo o desenvolvimento de métodos e algoritmos para projeto online de sistemas de controle ótimo. Esta abordagem é apresentada no contexto de programação dinâmica aproximada que permite aproximar a solução de realimentação ótima de modo a reduzir a complexidade computacional associada com métodos convencionais de programação dinâmica para controle ótimo de sistemas multivariáveis. Especi camente, no quadro de programação dinâmica heurística e programação dinâmica heurística dependente de ação, esta proposta é orientada para o desenvolvimento de soluções aproximadas online, numericamente estáveis, da equação de Hamilton-Jacobi-Bellman do tipo Riccati associada ao problema do regulador linear quadrático discreto que tem por base uma formulação que combina estimativas da função valor por meio de uma estrutura RLS (do inglês Recursive Least-Squares), diferenças temporais e melhorias de política. O desenvolvimento das metodologias propostas, neste trabalho, tem seu foco principal voltado para a fatoração UDU T que é inserida neste quadro para melhorar o processo de estimação RLS de políticas de decisão ótimas do regulador linear quadrá- tico discreto, contornando-se problemas de convergência e estabilidade numérica relacionados com o mal condicionamento da matriz de covariância da abordagem RLS.

Made available in DSpace on 2016-08-17T14:52:39Z (GMT). No. of bitstreams: 1 DISSERTACAO_DENIS FABRICIO SOUSA DE SA.pdf: 2409581 bytes, checksum: 4de5274676a1f75ffe2a1f6b46b1388c (MD5) Previous issue date: 2015-04-10 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / The representation of dynamic systems or plants via mathematical models occupies an important position in control system design that allow the performance evaluation of the controller during his development stage. These models are also used as an alternative to solve the problem of the hardness or impracticability to install sensors that measure the controlled variables, the dynamic systems representations enable non-invasive measurement of these variables. As consequence the designer has an alternative way to perform adaptive and optimal sensorless control for a given process. In this dissertation is presented a proposal for control systems schemas and algorithms, based on recurrent neural networks (ANN) and Box-Jenkins models, that are dedicated to sensorless or indirect control of dynamic systems. The proposed models and algorithms are associated with the systems identification and recurrent ANN approaches. The algorithms developed for the AAN training are Backpropagation Accelerated and RLS types that are compared with classical methods and strategies to obtain it online parameters of indirect control of system for a thermal plant, where the actuator is Peltier cell. The performance the parametric models of the plant and adaptive PID digital controllers and linear quadratic regulator (DLQR) that are the main elements of the sensorless temperature control system, are evaluated by means of hybrid simulations, where the algorithms implemented in micro controllers and the plant represented by mathematical models. The performance results of the proposed sensorless control algorithms are promissory, not only, in terms of the control system performance, but also due to the reexibility to deploy it in other dynamic systems. / A representação de sistemas dinâmicos ou plantas por meio modelos matemáticos ocupa uma posição relevante no projeto de sistemas de controle, permitindo que o projetista avalie o desempenho dos controladores durante a fase de desenvolvimento do projeto. Estes modelos também são utilizados para resolver o problema da dificuldade ou impossibilidade da inserção de sensores em plantas para medição de variáveis controladas, onde os modelos viabilizam a mediação não invasiva destas variáveis, fornecendo uma alternativa para realização do controle indireto adaptativo e ótimo de um dado processo. Nesta dissertação apresenta-se o desenvolvimento de modelos propostos baseados em redes neurais artificiais recorrentes para o controle sensorless ou indireto da planta. Os modelos propostos estão associados com as abordagens de Identificação de Sistemas e de RNA's recorrentes. OS algoritmos desenvolvidos para o treinamento das RNAs são do tipo Backpropagation acelerado e RLS, que são comparados com estratégias e métodos clássicos, para obtenção online dos parâmetros do sistema de controle indireto de uma planta térmica, tendo como atuador uma célula Peltier. Para uns de avaliação de desempenho do sistema de controle indireto da planta, os modelos paramétricos e controladores digitais adaptativos do tipo PID e regulador linear quadrático (DLQR) são avaliados por meio de simulações híbridas, sendo os algoritmos dos controladores implementados em microcontroladores e a planta representada por modelos matemáticos. Os resultados apresentados são promissores, não são sentido do desempenho do sistema de controle, mas também nos custos reduzidos para seu desenvolvimento, operação e flexibilidade de aplicação em outros sistemas dinâmicos.

Made available in DSpace on 2016-08-17T14:53:08Z (GMT). No. of bitstreams: 1 Marcio Mendes Cerqueira.pdf: 4042726 bytes, checksum: a65c6a7174271eecc553e3a5b0ceb33a (MD5) Previous issue date: 2010-02-05 / Conselho Nacional de Desenvolvimento Científico e Tecnológico / Motivated by the necessity of efficient algorithms, it s presented the development of a methodology for the design and analysis of state observers in open and closed loops that are dedicated to monitoring and control of dynamic systems. The development of observers are based on OE models, description in state space and Kalman filter. The models are evaluated for temperature control of a aluminum cube that is inside of a sterilizer oven. In addition to the models assessment in terms of its ability to represent behavior of plants, these models also evaluated for the design of discrete linear quadric regulator DLQR that are tuned by genetic algorithms. The monitoring models are evaluated for open and closed loops structures that are represented by algorithms in terms of difference equations, these algorithms are seen as software core for the indirect measurement systems. / Motivado pela necessidade de algoritmos eficientes, apresenta-se o desenvolvimento de uma metodologia para projeto e análise de observadores de estado em malhas aberta e fechada que são dedicados a monitoração e controle de sistemas dinâmicos. O desenvolvimento dos observadores estão fundamentados em modelos OE, descrição no espaço de estados e filtro de Kalman. Os modelos são avaliados para o controle da temperatura de um cubo de alumínio que encontra-se no interior de uma estufa. Além das avaliações dos modelos em termos de sua habilidade em representar comportamento de plantas, estes são também avaliados para o projeto do regulador linear quadrático discreto (RQLD) que são sintonizados por algoritmos genéticos. Aplicação dos modelos para monitoração é avaliada nas estruturas das malhas aberta e fechada que são representadas por algoritmos em da equação à diferença, tendo em vistas o desenvolvimento de núcleos de software para os sistemas de medição indireta.

Concorder un modèle à certaines observations, voilà qui résume assez bien ce que l’apprentissage machine cherche à accomplir. Ce concept est maintenant omniprésent dans nos vies, entre autre grâce aux percées récentes en apprentissage profond. La stratégie d’optimisation prédominante pour ces deux domaines est la minimisation d’un objectif donné. Et pour cela, la méthode du gradient, méthode de premier-ordre qui modifie les paramètres du modèle à chaque itération, est l’approche dominante. À l’opposé, les méthodes dites de second ordre n’ont jamais réussi à s’imposer en apprentissage profond. Pourtant, elles offrent des avantages reconnus qui soulèvent encore un grand intérêt. D’où l’importance de la méthode du col, qui unifie les méthodes de premier et second ordre sous un même paradigme. Dans ce mémoire, nous établissons un parralèle direct entre la méthode du col et le domaine du contrôle optimal ; domaine qui cherche à optimiser mathématiquement une séquence de décisions. Et certains des problèmes les mieux compris et étudiés en contrôle optimal sont les commandes linéaires quadratiques. Problèmes pour lesquels on connaît très bien la solution optimale. Plus spécifiquement, nous démontrerons l’équivalence entre une itération de la méthode du col et la résolution d’une Commande Linéaire Quadratique (CLQ). Cet éclairage nouveau implique une approche unifiée quand vient le temps de déployer nombre d’algorithmes issus de la méthode du col, tel que la méthode du gradient et celle des gradients naturels, sans être limitée à ceux-ci. Approche que nous étendons ensuite aux problèmes à horizon infini, tel que les modèles à équilibre profond. Ce faisant, nous démontrons pour ces problèmes que calculer les gradients via la différentiation implicite revient à employer l’équation de Riccati pour solutionner la CLQ associée à la méthode du gradient. Finalement, notons que l’incorporation d’information sur la courbure du problème revient généralement à rencontrer une inversion matricielle dans la méthode du col. Nous montrons que l’équivalence avec les CLQ permet de contourner cette inversion en utilisant une approximation issue des séries de Neumann. Surprenamment, certaines observations empiriques suggèrent que cette approximation aide aussi à stabiliser le processus d’optimisation quand des méthodes de second-ordre sont impliquées ; en agissant comme un régularisateur adaptif implicite. / Machine learning entails training a model to fit some given observations, and recent advances in the field, particularly in deep learning, have made it omnipresent in our lives. Fitting a model usually requires the minimization of a given objective. When it comes to deep learning, first-order methods like gradient descent have become a default tool for optimization in deep learning. On the other hand, second-order methods did not see widespread use in deep learning. Yet, they hold many promises and are still a very active field of research. An important perspective into both methods is steepest descent, which allows you to encompass first and second-order approaches into the same framework. In this thesis, we establish an explicit connection between steepest descent and optimal control, a field that tries to optimize sequential decision-making processes. Core to it is the family of problems known as Linear Quadratic Regulation; problems that have been well studied and for which we know optimal solutions. More specifically, we show that performing one iteration of steepest descent is equivalent to solving a Linear Quadratic Regulator (LQR). This perspective gives us a convenient and unified framework for deploying a wide range of steepest descent algorithms, such as gradient descent and natural gradient descent, but certainly not limited to. This framework can also be extended to problems with an infinite horizon, such as deep equilibrium models. Doing so reveals that retrieving the gradient via implicit differentiation is equivalent to recovering it via Riccati’s solution to the LQR associated with gradient descent. Finally, incorporating curvature information into steepest descent usually takes the form of a matrix inversion. However, casting a steepest descent step as a LQR also hints toward a trick that allows to sidestep this inversion, by leveraging Neumann’s series approximation. Empirical observations provide evidence that this approximation actually helps to stabilize the training process, by acting as an adaptive damping parameter.

Micro aerial vehicles and other autonomous systems have the potential to truly transform life as we know it, however much of the potential of autonomous systems remains unrealized because reliable navigation is still an unsolved problem with significant challenges. This dissertation presents solutions to many aspects of autonomous navigation. First, it presents ROSflight, a software and hardware architure that allows for rapid prototyping and experimentation of autonomy algorithms on MAVs with lightweight, efficient flight control. Next, this dissertation presents improvments to the state-of-the-art in optimal control of quadrotors by utilizing the error-state formulation frequently utilized in state estimation. It is shown that performing optimal control directly over the error-state results in a vastly more computationally efficient system than competing methods while also dealing with the non-vector rotation components of the state in a principled way. In addition, real-time robust flight planning is considered with a method to navigate cluttered, potentially unknown scenarios with real-time obstacle avoidance. Robust state estimation is a critical component to reliable operation, and this dissertation focuses on improving the robustness of visual-inertial state estimation in a filtering framework by extending the state-of-the-art to include better modeling and sensor fusion. Further, this dissertation takes concepts from the visual-inertial estimation community and applies it to tightly-coupled GNSS, visual-inertial state estimation. This method is shown to demonstrate significantly more reliable state estimation than visual-inertial or GNSS-inertial state estimation alone in a hardware experiment through a GNSS-GNSS denied transition flying under a building and back out into open sky. Finally, this dissertation explores a novel method to combine measurements from multiple agents into a coherent map. Traditional approaches to this problem attempt to solve for the position of multiple agents at specific times in their trajectories. This dissertation instead attempts to solve this problem in a relative context, resulting in a much more robust approach that is able to handle much greater intial error than traditional approaches.

This master thesis project is focused on the development of an LQR controller and its comparison with other controllers (PID and MPC), in order to successfully control an Autonomous Underwater Vehicle manipulation system. The modelling of the manipulator was performed first in Matlab and later on in Simulink-Simscape. Once the manipulator was integrated with the AUV model, the LQR controller was also developed initially in Matlab and then in Simulink. The controller was then extracted from Simulink as a C-code and verified in Stonefish. After confirming that the LQR code was working in Stonefish, its results from Simulink were compared with PID and MPC results for two different trajectories. The data for comparison and statistical analysis were divided into the two trajectory scenarios (horizontal and vertical) since the weight matrices of both controllers were different. Looking at the system’s overall behavior the Model Predictive Control (MPC) and LQR had similar results, regarding the rise time, overshoot, steady-state error and robustness to disturbances. An anticipated fact for the MPC was that it takes the longest run time for both scenarios. Lastly, as expected the PID had the worst response of all three controllers, in both scenarios. Implementing a PID on a nonlinear system, produced many oscillations without being able to stabilize at the reference value, thus giving a large steady-state error. In addition, it could not counteract the noise disturbances in the signal. / Detta examensarbete är inriktat på utvecklingen av en LQR-styrenhet och dess jämförelse med andra kontroller (PID och MPC), för att framgångsrikt styra ett autonomt undervattensfordon-manipulationssystem. Modelleringen av manipulatorn utfördes först i Matlab och senare i Simulink-Simscape. När manipulatorn väl hade integrerats med AUV modellen, utvecklades LQR styrenheten också inledningsvis i Matlab och sedan i Simulink. Kontrollenheten extraherades sedan från Simulink som en C-kod och verifierades i Stonefish. Efter att ha bekräftat att LQR koden fungerade i Stonefish, jämfördes resultaten från Simulink med PID och MPC resultat för två olika banor. Data för jämförelse och statistisk analys delades in i de två bana-scenarierna (horisontella och vertikala), eftersom viktmatriserna för båda kontrollerna var olika. När man tittar på systemets övergripande beteende hade Model Predictive Controller (MPC) och LQR liknande resultat när det gäller stigningstid, överskott, steady-state fel och robusthet mot störningar. Ett förväntat faktum för MPC var att det tar den längsta körtiden för båda scenarierna. Slutligen, som väntat, hade PID det sämsta svaret av alla tre kontrollerna, i båda scenarierna. Implementering av ett PID på ett olinjärt system gav många oscillationer utan att kunna stabilisera sig vid referensvärdet, vilket gav ett stort steady-state fel. Dessutom kunde den inte motverka bullerstörningarna i signalen.