The goal of this paper is to extend the results of Bayraktar and Young (2006) on minimizing an individual's probability of lifetime ruin; i.e. the probability that the individual goes bankrupt before dying. We consider a scenario in which the individual is allowed to invest in both a domestic bank account and a foreign bank account, with the exchange rate between the two currencies being modeled by geometric Brownian motion. Additionally, we impose the restriction that the individual is not allowed to borrow money, and assume that the individual's wealth is consumed at a constant rate. We derive formulas for the minimum probability of ruin as well as the individual's optimal investment strategy. We also give a few numerical examples to illustrate these results.

The object of this thesis is the study of some new financial models. The common feature is that they all involve optimal decisions. Some of the decisions take the form of a control and we enter the theory of stochastic optimal control and of Hamilton-Jacobi-Bellman (HJB) equations. Other decisions are "binary" and we deal with the theory of optimal stopping and free boundary problems. Throughout the thesis we will prefer a heuristic and intuitive approach to a too technical one which could hide the underlying ideas. In the first part we introduce the reader to option pricing, HJB equations and free boundary problems, and we review briefly the use of these mathematical tools in finance. The second part of the thesis deals with passport options. The pricing of these exotic options involves stochastic optimal control and free boundary problems. Finally, in the last part we study the end-of-the-year bonus for traders: how to optimally reward a trader?

This thesis introduces a selection of models for optimal execution of financial assets at the tactical level. As opposed to optimal scheduling, which defines a trading schedule for the trader, this thesis investigates how the trader should interact with the order book. If a trader is aggressive he will execute his order using market orders, which will negatively feedback on his execution price through market impact. Alternatively, the models we focus on consider a passive trader who places limit orders into the limit-order book and waits for these orders to be filled by market orders from other traders. We assume these models do not exhibit market impact. However, given we await market orders from other participants to fill our limit orders a new risk is borne: execution risk. We begin with an extension of Guéant et al. (2012b) who through the use of an exponential utility, standard Brownian motion, and an absolute decay parameter were able to cleverly build symmetry into their model which significantly reduced the complexity. Our model consists of geometric Brownian motion (and mean-reverting processes) for the asset price, a proportional control parameter (the additional amount we ask for the asset), and a proportional decay parameter, implying that the symmetry found in Guéant et al. (2012b) no longer exists. This novel combination results in asset-dependent trading strategies, which to our knowledge is a unique concept in this framework of literature. Detailed asymptotic analyses, coupled with advanced numerical techniques (informing the asymptotics) are exploited to extract the relevant dynamics, before looking at further extensions using similar methods. We examine our above mentioned framework, as well as that of Guéant et al. (2012), for a trader who has a basket of correlated assets to liquidate. This leads to a higher-dimensional model which increases the complexity of both numerically solving the problem and asymptotically examining it. The solutions we present are of interest, and comparable with Markowitz portfolio theory. We return to our framework of a single underlying and consider four extensions: a stochastic volatility model which results in an added dimension to the problem, a constrained optimisation problem in which the control has an explicit lower bound, changing the exponential intensity to a power intensity which results in a reformulation as a singular stochastic control problem, and allowing the trader to trade using both market orders and limit orders resulting in a free-boundary problem. We complete the study with an empirical analysis using limit-order book data which contains multiple levels of the book. This involves a novel calibration of the intensity functions which represent the limit-order book, before backtesting and analysing the performance of the strategies.

While stochastic optimal control, together with associate formulations like Reinforcement Learning, provides a formal approach to, amongst other, motor control, it remains computationally challenging for most practical problems. This thesis is concerned with the study of relations between stochastic optimal control and probabilistic inference. Such dualities { exempli ed by the classical Kalman Duality between the Linear-Quadratic-Gaussian control problem and the filtering problem in Linear-Gaussian dynamical systems { make it possible to exploit advances made within the separate fields. In this context, the emphasis in this work lies with utilisation of approximate inference methods for the control problem. Rather then concentrating on special cases which yield analytical inference problems, we propose a novel interpretation of stochastic optimal control in the general case in terms of minimisation of certain Kullback-Leibler divergences. Although these minimisations remain analytically intractable, we show that natural relaxations of the exact dual lead to new practical approaches. We introduce two particular general iterative methods ψ-Learning, which has global convergence guarantees and provides a unifying perspective on several previously proposed algorithms, and Posterior Policy Iteration, which allows direct application of inference methods. From these, practical algorithms for Reinforcement Learning, based on a Monte Carlo approximation to ψ-Learning, and model based stochastic optimal control, using a variational approximation of posterior policy iteration, are derived. In order to overcome the inherent limitations of parametric variational approximations, we furthermore introduce a new approach for none parametric approximate stochastic optimal control based on a reproducing kernel Hilbert space embedding of the control problem. Finally, we address the general problem of temporal optimisation, i.e., joint optimisation of controls and temporal aspects, e.g., duration, of the task. Specifically, we introduce a formulation of temporal optimisation based on a generalised form of the finite horizon problem. Importantly, we show that the generalised problem has a dual finite horizon problem of the standard form, thus bringing temporal optimisation within the reach of most commonly used algorithms. Throughout, problems from the area of motor control of robotic systems are used to evaluate the proposed methods and demonstrate their practical utility.

While the market for credit instruments grew continuously in the decade before 2008, its liquidity has dried up significantly in the current crisis, and investors have become aware of the possible consequences of being exposed to credit risk. In this thesis we address these issues by pricing credit instruments using utility indifference pricing, a method that takes into account the investor's personal risk aversion and which is not affected by the lack of liquidity. Through stochastic optimal control methods, we use indifference pricing with exponential utility to determine corporate bond prices and CDS spreads. In the first part we examine how these quantities are affected by risk aversion under different models of default. The emphasis lies on a hybrid model, in which a regime switch of the reference entity is triggered by a creditworthiness index correlated to its stock price. The second part generalizes this setup by introducing uncertainty in the model parameters. Robust optimal control has been used independently in the literature to address model uncertainty for portfolio selection problems. Here, we incorporate this approach with utility indifference and derive some analytical and numerical results on how model uncertainty affects credit spreads.

While the market for credit instruments grew continuously in the decade before 2008, its liquidity has dried up significantly in the current crisis, and investors have become aware of the possible consequences of being exposed to credit risk. In this thesis we address these issues by pricing credit instruments using utility indifference pricing, a method that takes into account the investor's personal risk aversion and which is not affected by the lack of liquidity. Through stochastic optimal control methods, we use indifference pricing with exponential utility to determine corporate bond prices and CDS spreads. In the first part we examine how these quantities are affected by risk aversion under different models of default. The emphasis lies on a hybrid model, in which a regime switch of the reference entity is triggered by a creditworthiness index correlated to its stock price. The second part generalizes this setup by introducing uncertainty in the model parameters. Robust optimal control has been used independently in the literature to address model uncertainty for portfolio selection problems. Here, we incorporate this approach with utility indifference and derive some analytical and numerical results on how model uncertainty affects credit spreads.

Motor noise and redundancy are vexing issues in motor control; yet their understanding provides great insights on underlying control mechanisms that govern movement. They provide glimpses into how the nervous system organizes and regulates movement within the motor control system. Understand of motor control could spur new advances in motor control could lead to better development of rehabilitation process and technology to counteract debilitating affects of neuromuscular disorders and motor readjustment with prostheses. However, before such process and technology could be developed and adapted for clinical use, a deeper understanding of motor control is needed to unravel the neural roadmap that regulates and generates movement. New theory of motor control could precipitate the development of more robust control mechanisms for robotic-human interaction. This work aims at expanding a more rigorous analytical and mathematical framework to understand how these control mechanisms reconcile redundancy and stochastic noise in human motor control. / text

This thesis solves estimation and control problems in computational neuroscience, mathematically dealing with the first-passage times of diffusion stochastic processes. We first derive estimation algorithms for model parameters from first-passage time observations, and then we derive algorithms for the control of first-passage times. Finally, we solve an optimal design problem which combines elements of the first two: we ask how to elicit first-passage times such as to facilitate model estimation based on said first-passage observations. The main mathematical tools used are the Fokker-Planck partial differential equation for evolution of probability densities, the Hamilton-Jacobi-Bellman equation of optimal control and the adjoint optimization principle from optimal control theory. The focus is on developing computational schemes for the solution of the problems. The schemes are implemented and are tested for a wide range of parameters.

Uruguay is a pioneer in the use of renewable sources of energy and can usually satisfy its total demand from renewable sources. Control and optimization of the system is complicated by half of the installed power - wind and solar sources - be- ing non-controllable with high uncertainty and variability. In this work we present a novel optimization technique for efficient use of the production facilities. The dy- namical system is stochastic, and we deal with its non-Markovian dynamics through a Lagrangian relaxation. Continuous-time optimal control and value function are found from the solution to a sequence of Hamilton-Jacobi-Bellman partial differential equations associated with the system. We introduce a monotone scheme to avoid spurious oscillations in the numerical solution and apply the technique to a number of examples taken from the Uruguayan grid. We use parallelization and change of variables to reduce the computational times. Finally, we study the usefulness of extra system storage capacity offered by batteries.

Esta tese trata do problema de controle ótimo estocástico de sistemas com saltos Markovianos e ruído multiplicativo a tempo discreto, com horizontes de tempo finito e infinito. A função custo é composta de termos quadráticos e lineares nas variáveis de estado e de controle, com matrizes peso indefinidas. Como resultado principal do problema com horizonte finito, é apresentada uma condição necessária e suficiente para que o problema de controle seja bem posto, a partir da qual uma solução ótima é derivada. A condição e a lei de controle são escritas em termos de um conjunto acoplado de equações de Riccati interconectadas a um conjunto acoplado de equações lineares recursivas. Para o caso de horizonte infinito, são apresentadas as soluções ótimas para os problemas de custo médio a longo prazo e com desconto, derivadas a partir de uma solução estabilizante de um conjunto de equações algébricas de Riccati acopladas generalizadas (GCARE). A existência da solução estabilizante é uma condição suficiente para que tais problemas sejam do tipo bem posto. Além disso, são apresentadas condições para a existência das soluções maximal e estabilizante do sistema GCARE. Como aplicações dos resultados obtidos, são apresentadas as soluções de um problema de otimização de carteiras de investimento com benchmark e de um problema de gestão de ativos e passivos de fundos de pensão do tipo benefício definido, ambos os casos com mudanças de regime nas variáreis de mercado. / This thesis considers the finite-horizon and infinite-horizon stochastic optimal control problem for discrete-time Markov jump with multiplicative noise linear systems. The performance criterion is assumed to be formed by a linear combination of a quadratic part and a linear part in the state and control variables. The weighting matrices of the state and control for the quadratic part are allowed to be indefinite. For the finite-horizon problem the main results consist of deriving a necessary and sufficient condition under which the problem is well posed and a optimal control law is derived. This condition and the optimal control law are written in terms of a set of coupled generalized Riccati difference equations interconnected with a set of coupled linear recursive equations. For the infinite-horizon problem a set of generalized coupled algebraic Riccati equations (GCARE) is studied. In this case, a sufficient condition under which there exists the maximal solution and a necessary and sufficient condition under which there exists the mean square stabilizing solution for the GCARE are presented. Moreover, a solution for the discounted and long run average cost problems is presented. The results obtained are applied to solver a portfolio optimization problem with benchmark and a pension fund problem with regime switching.