The solution of optimal control problems through the Hamilton-Jacobi-Bellman (HJB) equation offers guaranteed satisfaction of both the necessary and sufficient conditions for optimality. However, finding an exact solution to the HJB equation is a near impossible task for many optimal control problems. This thesis presents an approximation method for solving finite-horizon optimal control problems involving nonlinear dynamical systems. The method uses finite-order approximations of the partial derivatives of the cost-to-go function, and successive higher-order differentiations of the HJB equation. Natural byproducts of the proposed method provide sensitivities of the controls to changes in the initial states, which can be used to approximate the solution to neighboring optimal control problems. For highly nonlinear problems, the method is modified to calculate control sensitivities about a nominal trajectory. In this framework, the method is shown to provide accurate control sensitivities at much lower orders of approximation. Several numerical examples are presented to illustrate both applications of the approximation method.

The solution of optimal control problems through the Hamilton-Jacobi-Bellman (HJB) equation offers guaranteed satisfaction of both the necessary and sufficient conditions for optimality. However, finding an exact solution to the HJB equation is a near impossible task for many optimal control problems. This thesis presents an approximation method for solving finite-horizon optimal control problems involving nonlinear dynamical systems. The method uses finite-order approximations of the partial derivatives of the cost-to-go function, and successive higher-order differentiations of the HJB equation. Natural byproducts of the proposed method provide sensitivities of the controls to changes in the initial states, which can be used to approximate the solution to neighboring optimal control problems. For highly nonlinear problems, the method is modified to calculate control sensitivities about a nominal trajectory. In this framework, the method is shown to provide accurate control sensitivities at much lower orders of approximation. Several numerical examples are presented to illustrate both applications of the approximation method.

The Bolza-form of the finite-time constrained optimal control problem leads to the Hamilton-Jacobi-Bellman (HJB) equation with terminal boundary conditions and tobe- determined parameters. In general, it is a formidable task to obtain analytical and/or numerical solutions to the HJB equation. This dissertation presents two novel polynomial expansion methodologies for solving optimal feedback control problems for a class of polynomial nonlinear dynamical systems with terminal constraints. The first approach uses the concept of higher-order series expansion methods. Specifically, the Series Solution Method (SSM) utilizes a polynomial series expansion of the cost-to-go function with time-dependent coefficient gains that operate on the state variables and constraint Lagrange multipliers. A significant accomplishment of the dissertation is that the new approach allows for a systematic procedure to generate optimal feedback control laws that exactly satisfy various types of nonlinear terminal constraints. The second approach, based on modified Galerkin techniques for the solution of terminally constrained optimal control problems, is also developed in this dissertation. Depending on the time-interval, nonlinearity of the system, and the terminal constraints, the accuracy and the domain of convergence of the algorithm can be related to the order of truncation of the functional form of the optimal cost function. In order to limit the order of the expansion and still retain improved midcourse performance, a waypoint scheme is developed. The waypoint scheme has the dual advantages of reducing computational efforts and gain-storage requirements. This is especially true for autonomous systems. To illustrate the theoretical developments, several aerospace application-oriented examples are presented, including a minimum-fuel orbit transfer problem. Finally, the series solution method is applied to the solution of a class of partial differential equations that arise in robust control and differential games. Generally, these problems lead to the Hamilton-Jacobi-Isaacs (HJI) equation. A method is presented that allows this partial differential equation to be solved using the structured series solution approach. A detailed investigation, with several numerical examples, is presented on the Nash and Pareto-optimal nonlinear feedback solutions with a general terminal payoff. Other significant applications are also discussed for one-dimensional problems with control inequality constraints and parametric optimization.

We propose multigrid methods for solving Hamilton-Jacobi-Bellman (HJB) and Hamilton-Jacobi-Bellman-Isaacs (HJBI) equations. The methods are based on the full approximation scheme. We propose a damped-relaxation method as smoother for multigrid. In contrast with policy iteration, the relaxation scheme is convergent for both HJB and HJBI equations. We show by local Fourier analysis that the damped-relaxation smoother effectively reduces high frequency error. For problems where the control has jumps, restriction and interpolation methods are devised to capture the jump on the coarse grid as well as during coarse grid correction. We will demonstrate the effectiveness of the proposed multigrid methods for solving HJB and HJBI equations arising from option pricing as well as problems where policy iteration does not converge or converges slowly.

Η μερική διαφορική εξίσωση Hamilton-Jacobi-Bellman παράγει τη λύση στο πρόβλημα του υπολογισμού της βέλτιστης ανάδρασης καταστάσεων σε μη γραμμικά δυναμικά συστήματα. Η προσπάθεια ανάπτυξης εύχρηστων και αξιόπιστων μεθόδων αριθμητικής ή προσεγγιστικής επίλυσης της εξίσωσης Hamilton-Jacobi-Bellman έχει τεράστια σημασία στη ρύθμιση διεργασιών γιατί μπορεί να οδηγήσει άμεσα σε εργαλεία σχεδιασμού μη γραμμικών ρυθμιστών. Ειδικότερα, στη ρύθμιση διεργασιών, η απόδοση ενός ρυθμιστικού συστήματος αξιολογείται βάσει ενός τετραγωνικού δείκτη απόδοσης σε άπειρο χρονικό ορίζοντα, και η βέλτιστη ανάδραση καταστάσεων μπορεί να υπολογισθεί μέσω της λύσης της εξίσωσης Hamilton-Jacobi-Bellman, μη εξαρτώμενης από το χρόνο. Στο πρόβλημα της επίλυσης της παραπάνω εξίσωσης παρουσιάζονται σοβαρές δυσκολίες, κυρίως λόγω υπολογιστικής πολυπλοκότητας. Για το λόγο αυτό, οι μέχρι στιγμής πρακτικές εφαρμογές υπήρξαν περιορισμένες. Στην παρούσα εργασία αναπτύσσεται υπολογιστική μέθοδος, βασισμένη στον αλγόριθμο επαναλήψεων Newton-Kantorovich, η οποία επιτυγχάνει πολυωνυμική προσέγγιση της λύσης της μερικής διαφορικής εξίσωσης Hamilton-Jacobi-Bellman υπό μορφή αναπτύγματος σε δυναμοσειρά Taylor. Με τον τρόπο αυτό επιταχύνονται σημαντικά οι υπολογισμοί για τον προσδιορισμό της βέλτιστης ανάδρασης καταστάσεων. Η μέθοδος εφαρμόζεται αρχικά σε ένα παράδειγμα ισοθερμοκρασιακού αντιδραστήρα συνεχούς λειτουργίας με ανάδευση, ο οποίος παρουσιάζει δυναμική συμπεριφορά μη-ελάχιστης φάσης, με μία είσοδο, μία έξοδο και δύο καταστάσεις. Στη συνέχεια, εφαρμόζεται σε παραδείγματα μη ισοθερμοκρασιακού αντιδραστήρα αντίστοιχης δυναμικής συμπεριφοράς, τριών καταστάσεων, πρωτίστως με μία είσοδο και μία έξοδο και κατόπιν με δύο εισόδους και δύο εξόδους. Με ανάπτυξη και εφαρμογή κώδικα MAPLE για κάθε μία περίπτωση χωριστά, υπολογίζονται προσεγγιστικά οι βέλτιστοι νόμοι ανάδρασης και σχεδιάζονται οι βέλτιστες αποκρίσεις των εισόδων και των εξόδων κάθε ενός από τα παραπάνω συστήματα, ενώ ταυτόχρονα γίνεται και καταγραφή των αντίστοιχων χρόνων εκτέλεσης κάθε κώδικα. Τέλος, στην περίπτωση του ισοθερμοκρασιακού αντιδραστήρα, γίνεται σύγκριση της προτεινόμενης μεθόδου με προϋπάρχουσες, κατά κύριο λόγο σε ζητήματα χρόνων εκτέλεσης, αλλά και σε ζητήματα απόδοσης στη ρύθμιση. / The partial differential equation Hamilton-Jacobi-Bellman produces the solution in the problem of calculation of optimal state feedback in non-linear dynamic systems. The effort of designing functional and reliable, numerical or approximate, methods for solving Hamilton-Jacobi-Bellman equation has enormous importance in process control because it can lead directly to tools of planning non-linear regulators. More specifically, in process control, the attribution of a regulating system is evaluated using a quadratic performance index in infinite time horizon, and the optimal state feedback can be calculated by the solution of the non time depended Hamilton-Jacobi-Bellman equation. The problem of solving the equation above encounters serious difficulties, mainly because of the calculation complexity. For this reason, the practical applications existed until now were very few. In the present work a calculating method is developed, based in the iterative algorithm Newton-Kantorovich, which achieves polynomial approach of the solution of partial differential equation Hamilton-Jacobi-Bellman under the form of Taylor series expansion. Thus the calculations for the determination of optimal state feedback are considerably accelerated. The method is initially applied in an example of continuous stirred tank reactor, with non-minimum phase dynamic behavior, with one input, one output and two state variables. Afterwards, it is applied in examples of not isothermal reactor of the same dynamic behavior, three state variables, firstly with one input and one output variables and then with two input and two output variables. Using the symbolic program MAPLE, a code was developed for each case separately, which calculates approximately the optimal feedback laws and designs the optimal responses of the inputs and outputs of each of the systems above, while the corresponding times of implementation of each code are simultaneously recording. Finally, in the case of isothermal reactor, a comparison is made between the proposed and preexisting methods, mainly in the base of the time of implementations and the regulation performance.

We propose multigrid methods for solving Hamilton-Jacobi-Bellman (HJB) and Hamilton-Jacobi-Bellman-Isaacs (HJBI) equations. The methods are based on the full approximation scheme. We propose a damped-relaxation method as smoother for multigrid. In contrast with policy iteration, the relaxation scheme is convergent for both HJB and HJBI equations. We show by local Fourier analysis that the damped-relaxation smoother effectively reduces high frequency error. For problems where the control has jumps, restriction and interpolation methods are devised to capture the jump on the coarse grid as well as during coarse grid correction. We will demonstrate the effectiveness of the proposed multigrid methods for solving HJB and HJBI equations arising from option pricing as well as problems where policy iteration does not converge or converges slowly.

Cette thèse se développe sur deux fronts: nous nous concentrons sur les méthodes numériques des problèmes de contrôle optimal, en particulier sur le Principe de la Programmation Dynamique et sur le Model Predictive Control (MPC) et nous présentons des applications de techniques de contrôle en biologie. Dans la première partie, nous considérons l'approximation d'un problème de contrôle optimal avec horizon infini, qui combine une première étape, basée sur MPC permettant d'obtenir rapidement une bonne approximation de la trajectoire optimal, et une seconde étape, dans la quelle l¿équation de Bellman est résolue dans un voisinage de la trajectoire de référence. De cette façon, on peux réduire une grande partie de la taille du domaine dans lequel on résout l¿équation de Bellman et diminuer la complexité du calcul. Le deuxième sujet est le contrôle des méthodes Level Set: on considère un problème de contrôle optimal, dans lequel la dynamique est donnée par la propagation d'un graphe à une dimension, contrôlé par la vitesse normale. Un état finale est fixé, l'objectif étant de le rejoindre en minimisant une fonction coût appropriée. On utilise la programmation dynamique grâce à une réduction d'ordre de l'équation utilisant la Proper Orthogonal Decomposition. La deuxième partie est dédiée à l'application des méthodes de contrôle en biologie. On présente un modèle décrit par une équation aux dérivées partielles qui modélise l'évolution d'une population de cellules tumorales. On analyse les caractéristiques du modèle et on formule et résout numériquement un problème de contrôle optimal concernant ce modèle, où le contrôle représente la quantité du médicament administrée. / This thesis is divided in two parts: in the first part we focus on numerical methods for optimal control problems, in particular on the Dynamic Programming Principle and on Model Predictive Control (MPC), in the second part we present some applications of the control techniques in biology. In the first part of the thesis, we consider the approximation of an optimal control problem with an infinite horizon, which combines a first step based on MPC, to obtain a fast but rough approximation of the optimal trajectory and a second step where we solve the Bellman equation in a neighborhood of the reference trajectory. In this way, we can reduce the size of the domain in which the Bellman equation can be solved and so the computational complexity is reduced as well. The second topic of this thesis is the control of the Level Set methods: we consider an optimal control, in which the dynamics is given by the propagation of a one dimensional graph, which is controlled by the normal velocity. A final state is fixed and the aim is to reach the trajectory chosen as a target minimizing an appropriate cost functional. To apply the Dynamic Programming approach we firstly reduce the size of the system using the Proper Orthogonal Decomposition. The second part of the thesis is devoted to the application of control methods in biology. We present a model described by a partial differential equation that models the evolution of a population of tumor cells. We analyze the mathematical and biological features of the model. Then we formulate an optimal control problem for this model and we solve it numerically.

We propose a discontinuous Galerkin finite element method (DGFEM) for fully nonlinear elliptic Hamilton--Jacobi--Bellman (HJB) partial differential equations (PDE) of second order with Cordes coefficients. Our analysis shows that the method is both consistent and stable, with arbitrarily high-order convergence rates for sufficiently regular solutions. Error bounds for solutions with minimal regularity show that the method is generally convergent under suitable choices of meshes and polynomial degrees. The method allows for a broad range of hp-refinement strategies on unstructured meshes with varying element sizes and orders of approximation, thus permitting up to exponential convergence rates, even for nonsmooth solutions. Numerical experiments on problems with nonsmooth solutions and strongly anisotropic diffusion coefficients demonstrate the significant gains in accuracy and computational efficiency over existing methods. We then extend the DGFEM for elliptic HJB equations to a space-time DGFEM for parabolic HJB equations. The resulting method is consistent and unconditionally stable for varying time-steps, and we obtain error bounds for both rough and regular solutions, which show that the method is arbitrarily high-order with optimal convergence rates with respect to the mesh size, time-step size, and temporal polynomial degree, and possibly suboptimal by an order and a half in the spatial polynomial degree. Exponential convergence rates under combined hp- and τq-refinement are obtained in numerical experiments on problems with strongly anisotropic diffusion coefficients and early-time singularities. Finally, we show that the combination of a semismooth Newton method with nonoverlapping domain decomposition preconditioners leads to efficient solvers for the discrete nonlinear problems. The semismooth Newton method has a superlinear convergence rate, and performs very effectively in computations. We analyse the spectral bounds of nonoverlapping domain decomposition preconditioners for a model problem, where we establish sharp bounds that are explicit in both the mesh sizes and polynomial degrees. We then go beyond the model problem and show computationally that these algorithms lead to efficient and competitive solvers in practical applications to fully nonlinear HJB equations.

The presented thesis project is a study on train energy consumption calculation and optimal train driving strategies for minimum energy consumption. This study is divided into three parts; the first part is a proposed model for energy consumption calculation for trains based on driving resistances. The second part is a presentation of a method based on dynamic programming and the Hamilton-Jacobi-Bellman equation (Bellman’s backward approach) for obtaining optimal speed and control profiles leading to minimum energy consumption. The third part is a case study for a Bombardier Transportation case. It includes the presentation of a preliminary algorithm developed within this thesis project; an algorithm based on the HJB equation that can be further improved in order to be used online in real-time as an advisory system for train drivers.

Artificial vision is the problem of creating systems capable of processing visual information. A fundamental sub-problem of artificial vision is image segmentation, the problem of detecting a structure from a digital image. Examples of segmentation problems include the detection of a road from an aerial photograph or the determination of the boundaries of the brain's ventricles from medical imagery. The extraction of structures allows for subsequent higher-level cognitive tasks. One of them is shape comparison. For example, if the brain ventricles of a patient are segmented, can their shapes be used for diagnosis? That is to say, do the shapes of the extracted ventricles resemble more those of healthy patients or those of patients suffering from schizophrenia? This thesis deals with the problem of image segmentation and shape comparison in the mathematical framework of partial differential equations. The contribution of this thesis is threefold: 1. A technique for the segmentation of regions is proposed. A cost functional is defined for regions based on a non-parametric functional of the distribution of image intensities inside the region. This cost is constructed to favor regions that are homogeneous. Regions that are optimal with respect to that cost can be determined with limited user interaction. 2. The use of direction information is introduced for the segmentation of open curves and closed surfaces. A cost functional is defined for structures (curves or surfaces) by integrating a local, direction-dependent pattern detector along the structure. Optimal structures, corresponding to the best match with the pattern detector, can be determined using efficient algorithms. 3. A technique for shape comparison based on the Laplace equation is proposed. Given two surfaces, one-to-one correspondences are determined that allow for the characterization of local and global similarity measures. The local differences among shapes (resulting for example from a segmentation step) can be visualized for qualitative evaluation by a human expert. It can also be used for classifying shapes into, for example, normal and pathological classes.