This thesis is concerned with mathematical modelling and optimal control of constrained systems. Each of the systems under consideration is a system that can be controlled by one of the variables, and this control is subject to constraints. First, we consider middle-distance running where a runner's horizontal propulsive force is the control which is constrained to be within a given range. Middle-distance running is typically a strategy-intensive race as slipstreaming effects come into play since speeds are still relatively fast and runners can leave their starting lane. We formulate a two-runner coupled model and determine optimal strategies using optimal control theory. Second, we consider two applications of control systems with delay related to R&D expenditure. The first of these applications relates to the defence industry. The second relates to the pharmaceutical industry. Both applications are characterised by a long delay between initial investment in R&D and seeing the benefits of R&D realised. We formulate models tailored to each application and use optimal control theory to determine the optimal proportion of available funds to invest in R&D over a given time horizon. Third, we consider a mathematical model of urban burglary based on the Short model. We make some modifications to this model including the addition of deterrence due to police officer presence. Police officer density is the control variable, which is constrained due to a finite number of police officers. We look at different control strategies for the police and their effect on burglary hot-spot formation.

The problem of designing control systems for entire plants is studied. A review of previous works, available techniques and current research challenges is presented, followed by the description of some theoretical tools to improve plantwide control, including the proposal of an augmented lexicographic multi-objective optimization procedure. With these, an augmented hierarchical plantwide control design technique and an optimal multi-objective technique for integrated control structure selection and controller tuning are proposed. The main contributions of these proposed techniques are the inclusion of system identification and optimal control tuning as part of the plantwide design procedure for improved results, support to multi-objective control specifications and support to any type of plant and controllers. Finally, the proposed techniques are applied to industrial benchmarks to demonstrate and validate its applicability. / O problema de projetar sistemas de controle para plantas inteiras é estudado. Uma revisão de trabalhos anteriores, técnicas disponíveis e atuais desafios de pesquisa é apresentada, seguida da descrição de algumas ferramentas teóricas para melhorar o controle plantwide, incluindo a proposta de um procedimento de otimização multi-objetivo lexicográfico aumentado. Com tais elementos, são propostas uma nova técnica hierárquica aumentada de projeto de sistemas de controle plantwide e uma técnica multi-objetivo para seleção de estrutura de controlador integrada à sintonia ótima do controlador. As principais contribuições das técnicas propostas são a inclusão de identificação de sistemas e sintonia ótima de controladores como parte do procedimento de projeto de controle plantwide para melhores resultados, suporte a especificações multi-objetivo e suporte a quaisquer tipos de plantas e controladores. Finalmente, as técnicas propostas são aplicadas a benchmarks industriais para demonstrar e validar sua aplicabilidade.

The primary goal of this thesis is to study finite element based a priori and a posteriori error estimates of optimal control problems of various kinds governed by linear elliptic PDEs (partial differential equations) of second and fourth orders. This thesis studies interior and boundary control (Neumann and Dirichlet) problems. The initial chapter is introductory in nature. Some preliminary and fundamental results of finite element methods and optimal control problems which play key roles for the subsequent analysis are reviewed in this chapter. This is followed by a brief literature survey of the finite element based numerical analysis of PDE constrained optimal control problems. We conclude the chapter with a discussion on the outline of the thesis. An abstract framework for the error analysis of discontinuous Galerkin methods for control constrained optimal control problems is developed in the second chapter. The analysis establishes the best approximation result from a priori analysis point of view and delivers a reliable and efficient a posteriori error estimator. The results are applicable to a variety of problems just under the minimal regularity possessed by the well-posedness of the problem. Subsequently, the applications of p p - interior penalty methods for a boundary control problem as well as a distributed control problem governed by the bi-harmonic equation subject to simply supported boundary conditions are discussed through the abstract analysis. In the third chapter, an alternative energy space based approach is proposed for the Dirichlet boundary control problem and then a finite element based numerical method is designed and analyzed for its numerical approximation. A priori error estimates of optimal order in the energy norm and the m norm are derived. Moreover, a reliable and efficient a posteriori error estimator is derived with the help an auxiliary problem. An energy space based Dirichlet boundary control problem governed by bi-harmonic equation is investigated and subsequently a l y - interior penalty method is proposed and analyzed for it in the fourth chapter. An optimal order a priori error estimate is derived under the minimal regularity conditions. The abstract error estimate guarantees optimal order of convergence whenever the solution has minimum regularity. Further an optimal order l l norm error estimate is derived. The fifth chapter studies a super convergence result for the optimal control of an interior control problem with Dirichlet cost functional and governed by second order linear elliptic PDE. An optimal order a priori error estimate is derived and subsequently a super convergence result for the optimal control is derived. A residual based reliable and efficient error estimators are derived in a posteriori error control for the optimal control. Numerical experiments illustrate the theoretical results at the end of every chapter. We conclude the thesis stating the possible extensions which can be made of the results presented in the thesis with some more problems of future interest in this direction.

The problem of designing control systems for entire plants is studied. A review of previous works, available techniques and current research challenges is presented, followed by the description of some theoretical tools to improve plantwide control, including the proposal of an augmented lexicographic multi-objective optimization procedure. With these, an augmented hierarchical plantwide control design technique and an optimal multi-objective technique for integrated control structure selection and controller tuning are proposed. The main contributions of these proposed techniques are the inclusion of system identification and optimal control tuning as part of the plantwide design procedure for improved results, support to multi-objective control specifications and support to any type of plant and controllers. Finally, the proposed techniques are applied to industrial benchmarks to demonstrate and validate its applicability. / O problema de projetar sistemas de controle para plantas inteiras é estudado. Uma revisão de trabalhos anteriores, técnicas disponíveis e atuais desafios de pesquisa é apresentada, seguida da descrição de algumas ferramentas teóricas para melhorar o controle plantwide, incluindo a proposta de um procedimento de otimização multi-objetivo lexicográfico aumentado. Com tais elementos, são propostas uma nova técnica hierárquica aumentada de projeto de sistemas de controle plantwide e uma técnica multi-objetivo para seleção de estrutura de controlador integrada à sintonia ótima do controlador. As principais contribuições das técnicas propostas são a inclusão de identificação de sistemas e sintonia ótima de controladores como parte do procedimento de projeto de controle plantwide para melhores resultados, suporte a especificações multi-objetivo e suporte a quaisquer tipos de plantas e controladores. Finalmente, as técnicas propostas são aplicadas a benchmarks industriais para demonstrar e validar sua aplicabilidade.

<p>Numerical trajectory optimization is helpful not only for mission planning but also design</p> <p>space exploration and quantifying vehicle performance. Direct methods for solving the opti-</p> <p>mal control problems, which first discretize the problem before applying necessary conditions</p> <p>of optimality, dominate the field of trajectory optimization because they are easier for the</p> <p>user to set up and are less reliant on a forming a good initial guess. On the other hand,</p> <p>many consider indirect methods, which apply the necessary conditions of optimality prior to</p> <p>discretization, too difficult to use for practical applications. Indirect methods though provide</p> <p>very high quality solutions, easily accessible sensitivity information, and faster convergence</p> <p>given a sufficiently good guess. Those strengths make indirect methods especially well-suited</p> <p>for generating large data sets for system analysis and worth revisiting.</p> <p>Recent advancements in the application of indirect methods have already mitigated many</p> <p>of the often cited issues. Automatic derivation of the necessary conditions with computer</p> <p>algebra systems have eliminated the manual step which was time-intensive and error-prone.</p> <p>Furthermore, regularization techniques have reduced problems which traditionally needed</p> <p>many phases and complex staging, like those with inequality path constraints, to a signifi-</p> <p>cantly easier to handle single arc. Finally, continuation methods can circumvent the small</p> <p>radius of convergence of indirect methods by gradually changing the problem and use previ-</p> <p>ously found solutions for guesses.</p> <p>The new optimal control problem solver Giuseppe incorporates and builds upon these</p> <p>advancements to make indirect methods more accessible and easily used. It seeks to enable</p> <p>greater research and creative approaches to problem solving by being more flexible and</p> <p>extensible than previous solvers. The solver accomplishes this by implementing a modular</p> <p>design with well-defined internal interfaces. Moreover, it allows the user easy access to and</p> <p>manipulation of component objects and functions to be use in the way best suited to solve</p> <p>a problem.</p> <p>A new technique simplifies and automates what was the predominate roadblock to using</p> <p>continuation, the generation of an initial guess for the seed solution. Reliable generation of</p> <p>a guess sufficient for convergence still usually required advanced knowledge optimal contrtheory or sometimes incorporation of an entirely separate optimization method. With the</p> <p>new method, a user only needs to supply initial states, a control profile, and a time-span</p> <p>over which to integrate. The guess generator then produces a guess for the “primal” problem</p> <p>through propagation of the initial value problem. It then estimates the “dual” (adjoint)</p> <p>variables by the Gauss-Newton method for solving the nonlinear least-squares problem. The</p> <p>decoupled approach prevents poorly guessed dual variables from altering the relatively easily</p> <p>guess primal variables. As a result, this method is simpler to use, faster to iterate, and much</p> <p>more reliable than previous guess generation techniques.</p> <p>Leveraging the continuation process also allows for greater insight into the solution space</p> <p>as there is only a small marginal cost to producing an additional nearby solutions. As a</p> <p>result, a user can quickly generate large families of solutions by sweeping parameters and</p> <p>modifying constraints. These families provide much greater insight in the general problem</p> <p>and underlying system than is obtainable with singular point solutions. One can extend</p> <p>these analyses to high-dimensional spaces through construction of compound continuation</p> <p>strategies expressible by directed trees.</p> <p>Lastly, a study into common convergence explicates their causes and recommends mitiga-</p> <p>tion strategies. In this area, the continuation process also serves an important role. Adaptive</p> <p>step-size routines usually suffice to handle common sensitivity issues and scaling constraints</p> <p>is simpler and out-performs scaling parameters directly. Issues arise when a cost functional</p> <p>becomes insensitive to the control, which a small control cost mitigates. The best perfor-</p> <p>mance of the solver requires proper sizing of the smoothing parameters used in regularization</p> <p>methods. An asymptotic increase in the magnitude of adjoint variables indicate approaching</p> <p>a feasibility boundary of the solution space.</p> <p>These techniques for indirect methods greatly facilitate their use and enable the gen-</p> <p>eration of large libraries of high-quality optimal trajectories for complex problems. In the</p> <p>future, these libraries can give a detailed account of vehicle performance throughout its flight</p> <p>envelope, feed higher-level system analyses, or inform real-time control applications.</p>

This master thesis focuses on developing a Reinforcement Learning (RL) controller to perform hydrobatic maneuvers on an Autonomous Underwater Vehicle (AUV) successfully. This work also aims to analyze the robustness of the RL controller, as well as provide a comparison between RL algorithms and Proportional Integral Derivative (PID) control. Training of the algorithms is initially conducted in a Numpy simulation in Python. We show how to model the Equations of Motion (EOM) of the AUV and how to use it to train the RL controllers. We use the stablebaselines3 RL framework and create a training environment with the OpenAI gym. The Twin-Delay Deep Deterministic Policy Gradient (TD3) algorithm offers good performance in the simulation. The following maneuvers are studied: trim control, waypoint following, and an inverted pendulum. We test the maneuvers both in the Numpy simulation and Stonefish simulator. Also, we test the robustness of the RL trim controller by simulating noise in the state feedback. Lastly, we run the RL trim controller on a real AUV hardware called SAM. We show that the RL algorithm trained in the Numpy simulator can achieve similar performance to the PID controller in the Stonefish simulator. We generate a policy that can perform the trim control and the Inverted Pendulum maneuver in the Numpy simulation. We show that we can generate a robust policy that executes other types of maneuvers by providing a parameterized cost function to the RL algorithm. We discuss the results of every maneuver we perform with the SAM AUV and provide a discussion about the advantages and disadvantages of this control method applied to underwater robotics. We conclude that RL can be used to create policies that perform hydrobatic maneuvers. This data-driven approach can be applied in the future to more complex problems in underwater robotics. / Denna masteruppsats fokuserar på att utveckla en Reinforcement Learning (RL) kontroller för att framgångsrikt utföra hydrobatiska manövrar på ett autonomt undervattensfordon (AUV). Detta arbete syftar också till att analysera robustheten hos RL-kontrollern, samt tillhandahålla en jämförelse mellan RL-algoritmer och Proportional Integral Derivative (PID) kontroll. Träning av algoritmerna utförs initialt i Numpy-simuleringen i Python. Vi visar hur man modellerar rörelseekvationerna (EOM) för AUV, och hur man använder den för att träna RL-kontrollerna. Vi använder ramverket stablebaselines3 RL och skapar en träningsmiljö med gymmet OpenAI. Algoritmen Twin-Delay Deep Deterministic Policy Gradient (TD3) erbjuder bra prestanda i simuleringen. Följande manövrar studeras: trimkontroll, waypointföljning och en inverterad pendel. Vi testar manövrarna både i Numpy-simulering och Stonefish-simulator. Vi testar också robustheten hos RL-trimkontrollern genom att simulera bruset i tillståndsåterkopplingen. Slutligen kör vi RL-trimkontrollern på den riktiga SAM AUV-hårdvaran. Vi visar att RL-algoritmen tränad i Numpy-simulatorn kan uppnå liknande prestanda som PID-regulatorn i Stonefish-simulatorn. Vi genererar en policy som kan utföra trimkontrollen och manövern med inverterad pendel i Numpy-simuleringen. Vi visar att vi kan generera en robust policy som utför andra typer av manövrar genom att tillhandahålla en parameteriserad kostnadsfunktion till RL-algoritmen. Vi diskuterar resultaten av varje manöver vi utför med SAM AUV och ger en diskussion om fördelarna och nackdelarna med denna kontrollmetod som tillämpas på undervattensrobotik. Vi drar slutsatsen att RL kan användas för att skapa policyer som utför hydrobatiska manövrar. Detta datadrivna tillvägagångssätt kan tillämpas i framtiden på mer komplexa problem inom undervattensrobotik.

Doctor of Philosophy / Department of Electrical and Computer Engineering / Balasubramaniam Natarajan / For any automated surveillance operation to be successful, it is critical to have sensing resources strategically positioned to observe, interpret, react and maybe even predict events.In many practical scenarios, it is also expected that different zones within a surveillance area may have different probability of event detection (or false alarm) requirements. The operational objective in such surveillance systems is to optimize resources (number of sensors and the associated cost) and their deployment while guaranteeing a certain assured level of detection/false alarm performance. In this dissertation, we study two major challenges related to sensor deployment in distributed sensor networks (DSNs) for detection applications. The first problem we study is the sensor deployment problem in which we ask the following question: Given a finite number of sensors (with a known sensing profile), how can we deploy these sensors such that we best meet the detection and false alarm requirements in a DSN employing a specific information fusion rule? Even though sensor deployment has garnered significant interest in the past, a unified, analytical framework to model and study sensor deployment is lacking. Additionally, the algorithms proposed in literature are typically heuristic in nature and are limited to (1) simplistic DSN fusion architectures, and (2) DSNs with uniform detection/false alarm requirements. In this dissertation, we propose a novel treatment of the sensor deployment problem using concepts from optimal control theory. Specifically, the deployment problem is formulated as a linear quadratic regulator (LQR) problem which provides a rigorous and analytical framework to study the deployment problem. We develop new sensor deployment algorithms that are applicable to a wide range of DSN architectures employing different fusion rules such as (1) logical OR fusion; (2) value fusion; (3) majority decision fusion, and (4) optimal decision fusion. In all these cases, we demonstrate that our proposed control theoretic deployment approach is able to significantly outperform previously proposed algorithms. The second problem considered in this dissertation is the “self healing” problem in which we ask the following question: After the failure of a number of sensors, how can one reconfigure the DSN such that the performance degradation due to sensor loss is minimized? Prior efforts in tackling the self healing problem typically rely on assumptions that don’t accurately capture the behavior of practical sensors/networks and focus on minimizing performance degradation at a local area of the network instead of considering overall performance of the DSN. In this work, we propose two self healing strategies the first approach relies on adjusting decision thresholds at the fusion center. The second approach involves sensor redeployment based on our control theoretic deployment framework. Simulation results illustrate that the proposed algorithms are effective in alleviating the performance degradation due to sensor loss.

Fokker-Planck equations, along with stochastic differential equations, play vital roles in physics, population modeling, game theory and optimization (finite or infinite dimensional). In this thesis, we study three topics, both theoretically and computationally, centered around them. In part one, we consider the optimal transport for finite discrete states, which are on a finite but arbitrary graph. By defining a discrete 2-Wasserstein metric, we derive Fokker-Planck equations on finite graphs as gradient flows of free energies. By using dynamical viewpoint, we obtain an exponential convergence result to equilibrium. This derivation provides tools for many applications, including numerics for nonlinear partial differential equations and evolutionary game theory. In part two, we introduce a new stochastic differential equation based framework for optimal control with constraints. The framework can efficiently solve several real world problems in differential games and Robotics, including the path-planning problem. In part three, we introduce a new noise model for stochastic oscillators. With this model, we prove global boundedness of trajectories. In addition, we derive a pair of associated Fokker-Planck equations.

Continuous Descent Arrival (CDA) is a procedure where aircraft descend, at or near idle thrust, from their cruise altitude to their Final Approach Fix without leveling off. By eliminating inefficient leveling off at low altitude, CDA provides benefits such as fuel savings, flight time savings, and the significant noise reduction near airports, but the usage of CDAs has been limited in low traffic condition due to difficulty in the separation management. For the successful CDA without degradation of the runway throughput, air traffic controllers should know the performance bound of the CDA trajectory and control the time of arrival for each aircraft, which is interpreted as Required Time of Arrival (RTA) from the aircraft standpoint. This thesis proposes a novel trajectory optimization methodology to meet RTA constraint. The CDA trajectory optimization problem in the flight management system is modeled as a path constrained optimal control problem of switched dynamical system. A sequential method that performs mode sequence estimation and parameter optimization, sequentially, is proposed to solve this problem. By analyzing the relaxed optimal solution with simplified dynamics, a computationally efficient algorithm to find the optimal switching structure is proposed and applied for the mode sequence estimation. This thesis also proposes a performance-bound analysis methodology using optimal control techniques to help controllers make a feasible schedule for CDA operations at a meter fix. The feasible time range analysis for a wide variety of aircraft is performed by using the proposed methodology. Based on the analysis result, a single flight time strategy is proposed for the application of CDA in high traffic conditions. The simulation with real traffic data has been shown that the single flight time strategy, combined with the proposed fixed RTA trajectory optimization, guarantees the conflict free CDA operation.

<p>Constrained optimization is used to derive human postures and movements. In the first study a static 3D model with 30 muscle groups is used to analyse postures. The activation levels of these muscles are minimized in order to represent the individual's choice of posture. Subject specific data in terms of anthropometry, strength and orthopedic aids serve as input. The aim is to study effects from orthopedic treatment and altered abilities of the subject. Initial validation shows qualitative agreement of posture strategies but further details about passive stiffness and anthropometry are needed, especially to predict pelvis orientation. In the second application, the athletic long jump, a problem formulation is developed to find optimal movements of a multibody system when subjected to contact. The model was based on rigid links, joint actuators and a wobbling mass. The contact to the ground was modelled as a spring-damper system with tuned properties. The movement in the degrees of freedom representing physical joints was described over contact time through two fifth-order polynomials, with a variable transition time, while the motion in the degrees of freedom of contact and wobbling mass was integrated forwards in time, as a consequence. Muscle activation variables were then optimized in order to maximize ballistic flight distance. The optimization determined contact time, end configuration, activation and interaction with the ground from an initial configuration. The results from optimization show a reasonable agreement with experimentally recorded jumps, but individual recordings and measurements are needed for more precise conclusions.</p><p> </p>