<div>Reinforcement Learning (RL) is a data-driven technique which is finding increasing application in the development of controllers for sequential decision making problems. Their wide adoption can be attributed to the fact that the development of these controllers is independent of the</div><div>knowledge of the system and thus can be used even when the environment dynamics are unknown. Model-Based RL controllers explicitly model the system dynamics from the observed (training) data using a function approximator, followed by using a path planning algorithm to obtain the optimal control sequence. While these controllers have been proven to be successful in simulations, lack of strong safety guarantees in the presence of noise makes them ill-posed for deployment on hardware, specially in safety critical systems. The proposed work aims at bridging this gap by providing a verification framework to evaluate the safety guarantees for a Model-Based RL controller. Our method builds upon reachability analysis to determine if there is any action which can drive the system into a constrained (unsafe) region. Consequently, our method can provide a binary yes or no answer to whether all the initial set of states are (un)safe to propagate trajectories from in the presence of some bounded noise.</div>

Uncertainties and constraints are present in most control systems. For example, robot motion planning and building climate regulation can be modeled as uncertain constrained systems. In this thesis, we develop mathematical and computational tools to analyze and synthesize controllers for such systems. As our first contribution, we characterize when a set is a probabilistic controlled invariant set and we develop tools to compute such sets. A probabilistic controlled invariantset is a set within which the controller is able to keep the system state with a certainprobability. It is a natural complement to the existing notion of robust controlled invariantsets. We provide iterative algorithms to compute a probabilistic controlled invariantset within a given set based on stochastic backward reachability. We prove that thesealgorithms are computationally tractable and converge in a finite number of iterations. The computational tools are demonstrated on examples of motion planning, climate regulation, and model predictive control. As our second contribution, we address the control design problem for uncertain constrained systems with aperiodic sensing and actuation. Firstly, we propose a stochastic self-triggered model predictive control algorithm for linear systems subject to exogenous disturbances and probabilistic constraints. We prove that probabilistic constraint satisfaction, recursive feasibility, and closed-loop stability can be guaranteed. The control algorithm is computationally tractable as we are able to reformulate the problem into a quadratic program. Secondly, we develop a robust self-triggered control algorithm for time-varying and uncertain systems with constraints based on reachability analysis. In the particular case when there is no uncertainty, the design leads to a control system requiring minimum number of samples over finite time horizon. Furthermore, when the plant is linear and the constraints are polyhedral, we prove that the previous algorithms can be reformulated as mixed integer linear programs. The method is applied to a motion planning problem with temporal constraints. / <p>QC 20181016</p>

This thesis is concerned with the Pursuit-and-Evasion (PE) problem where the pursuer aims to minimize the time to capture the evader while the evader tries to prevent capture. In the problem, the evader has two advantages: a higher manoeuvrability and that the pursuer is uncertain about the evader??s state. Cooperation among multiple pursuer vehicles can thus be used to overcome the evader??s advantages. The focus here is on the formulation and development of frameworks and algorithms for cooperation amongst pursuers, aiming at feasible implementation on real and autonomous vehicles. The thesis is split into Parts I and II. Part I considers the problem of capturing an evader of higher manoeuvrability in a deterministic PE game. The approach is the employment of Forward Reachable Set (FRS) analysis in the pursuers?? control. The analysis considers the coverage of the evader??s FRS, which is the set of reachable states at a future time, with the pursuer??s FRS and assumes that the chance of capturing the evader is dependent on the degree of the coverage. Using the union of multiple pursuers?? FRSs intuitively leads to more evader FRS coverage and this forms the mechanism of cooperation. A framework for cooperative control based on the FRS coverage, or FRS-based control, is proposed. Two control algorithms were developed within this framework. Part II additionally introduces the problem of evader state uncertainty due to noise and limited field-of-view of the pursuers?? sensors. A search-and-capture (SAC) problem is the result and a hybrid architecture, which includes multi-sensor estimation using the Particle Filter as well as FRS-based control, is proposed to accomplish the SAC task. The two control algorithms in Part I were tested in simulations against an optimal guidance algorithm. The results show that both algorithms yield a better performance in terms of time and miss distance. The results in Part II demonstrate the effectiveness of the hybrid architecture for the SAC task. The proposed frameworks and algorithms provide insights for the development of effective and more efficient control of pursuer vehicles and can be useful in the practical applications such as defence systems and civil law enforcement.

This thesis is concerned with the Pursuit-and-Evasion (PE) problem where the pursuer aims to minimize the time to capture the evader while the evader tries to prevent capture. In the problem, the evader has two advantages: a higher manoeuvrability and that the pursuer is uncertain about the evader??s state. Cooperation among multiple pursuer vehicles can thus be used to overcome the evader??s advantages. The focus here is on the formulation and development of frameworks and algorithms for cooperation amongst pursuers, aiming at feasible implementation on real and autonomous vehicles. The thesis is split into Parts I and II. Part I considers the problem of capturing an evader of higher manoeuvrability in a deterministic PE game. The approach is the employment of Forward Reachable Set (FRS) analysis in the pursuers?? control. The analysis considers the coverage of the evader??s FRS, which is the set of reachable states at a future time, with the pursuer??s FRS and assumes that the chance of capturing the evader is dependent on the degree of the coverage. Using the union of multiple pursuers?? FRSs intuitively leads to more evader FRS coverage and this forms the mechanism of cooperation. A framework for cooperative control based on the FRS coverage, or FRS-based control, is proposed. Two control algorithms were developed within this framework. Part II additionally introduces the problem of evader state uncertainty due to noise and limited field-of-view of the pursuers?? sensors. A search-and-capture (SAC) problem is the result and a hybrid architecture, which includes multi-sensor estimation using the Particle Filter as well as FRS-based control, is proposed to accomplish the SAC task. The two control algorithms in Part I were tested in simulations against an optimal guidance algorithm. The results show that both algorithms yield a better performance in terms of time and miss distance. The results in Part II demonstrate the effectiveness of the hybrid architecture for the SAC task. The proposed frameworks and algorithms provide insights for the development of effective and more efficient control of pursuer vehicles and can be useful in the practical applications such as defence systems and civil law enforcement.

Safety verification of continuous dynamical systems require the computationof the reachable set. The reachable set comprises those states the systemcan reach at a specific point in time. The present work aims to compute thisreachable set for the marine vessel, in the presence of uncertainties in thedynamic modeling of the system and in the presence of external disturbancesin the form of wind, waves and currents. The reachable set can then be usedto check if the vessel collides with an obstacle. The dynamic model used isthat of a nonlinear maneuvering model for the marine vessel. The dynamicson the azipod actuators are also considered.Several methods are considered to solve the reachability problem for themarine vessel. The first method considered is that of the Hamilton JacobiReachability analysis, where a dynamic game between the control input andthe disturbance input is played. This results in a dynamic programmingproblem known as the Hamilton Jacobi Bellman Isaacs (HJBI) equation. Itis solved using the Level-Set method, but it suffers from the curse of dimensionality.The other method considered is the use of set-theoretic approach,where an over-approximation of the reachable set is computed, in the contextof safety verification. But on the downside, large sets of admissible controlyields highly over-approximated reachable sets, which cannot be usedIn order to overcome the disadvantages posed by the first two methods,emphasizing on the real-time computation, a third method is developed, wherea supervised classification algorithm is used to compute the reachable setboundary. The dataset required for the classification algorithm is computedby solving a 2 Point Boundary Value Optimal Control Problem for the marinevessel. The features for classification algorithm can be extended, so as toinclude the uncertainties and disturbances in the system. The computationtime is greatly reduced and the accuracy of the method is comparable to theexact reachable set computation. / Säkerhetsverifiering av kontinuerliga dynamiska system kräver beräkningav mängden av tillstånd som kan nås vid en specifik tidpunkt, givet dess initialtillstånd.Detta arbete fokuserar påatt bestämma denna mängd av nåbaratillstånd för ett marint fartyg under modellosäkerheter och externa störningari form av vind, vågor och strömmar. Den nåbara mängden av tillstånd användssedan för att kontrollera om fartyget riskerar att kollidera med hinder.Den dynamiska modell som används i våra studier är en icke-linjär modelldär även dynamiken hos azipod-ställdonen betraktas.Arbetet studerar flera metoder för att lösa problemet: en klassisk Hamilton-Jacobi nåbarhetsanalys, en mängd-teoretisk teknik, samt en ny metod baseradpåmaskininlärning. Numeriska simuleringsstudier bekräftar att den föreslagnamaskininlärningsmetoden är snabbare än de tvåalternativen.

Dans cette thèse nous étudions la dynamique engendrée par une famille de flots Hamiltoniens. Un tel système dynamique à plusieurs générateurs est aussi appelé ‘polysystème’. Motivés par des questions liées au phénomène de la diffusion d’Arnold, notre objectif est de construire des trajectoires du polysystème qui relient deux régions lointaines de l’espace des phases. La thèse est divisée en trois parties.Dans la première partie, nous considérons le polysystème engendré par les flots discrétisés d’une famille d’Hamiltoniens Tonelli. En utilisant une approche variationnelle issue de la théorie KAM faible, nous donnons des conditions suffisantes pour l’existence des trajectoires souhaitées.Dans la deuxième partie, nous traitons le cas d’un polysystème engendré par un couple de flots Hamiltoniens à temps continu, dont l’étude rentre dans le cadre de la théorie géométrique du contrôle. Dans ce contexte, nous montrons dans certains cas la transitivité d’un polysystème générique, à l’aide du théorème de transversalité de Thom.La dernière partie de la thèse est dédiée à obtenir une nouvelle version du théorème de transversalité de Thom s’exprimant en termes d’ensembles rectifiables de codimension positive. Dans cette partie il n’est pas question de polysystèmes, ni d’Hamiltoniens. Néanmoins, les résultats obtenus ici sont utilisés dans la deuxième partie de la thèse / In this thesis we study the dynamics generated by a family of Hamiltonian flows. Such a dynamical system with several generators is also called ‘polysystem’.Motivated by some questions related to the phenomenon of Arnold diffusion, our aim is to construct trajectories of the polysystem which connect two far-apart regions of the phase space.The thesis is divided into three parts.In the first part, we consider the polysystem generated by the time-onemaps of a family of Tonelli Hamiltonians. By using a variational approach falling within the framework of weak KAM theory, we give sufficient conditions for the existence of the desired trajectories.In the second part, we address the case of a polysystem generated by twocontinuous-time Hamiltonian flows. This problem fits into the framework of geometriccontrol theory. In this context, we show in some cases the transitivity of a generic polysystem, by means of Thom’s transversality theorem.The third and last part of the thesis is devoted to the proof of a newversion of Thom’s transversality theorem, formulated in terms of rectifiable sets of positive codimension. Neither polysystems nor Hamiltonians are explicitly involved in this part. However, the results obtained here are used in the second part of the thesis.