We revisit the optimal investment and consumption model of Davis and Norman (1990) and Shreve and Soner (1994), following a shadow-price approach similar to that of Kallsen and Muhle-Karbe (2010). Making use of the completeness of the model without transaction costs, we reformulate and reduce the Hamilton-Jacobi-Bellman equation for this singular stochastic control problem to a non-standard free-boundary problem for a first-order ODE with an integral constraint. Having shown that the free boundary problem has a smooth solution, we use it to construct the solution of the original optimal investment/consumption problem in a self-contained manner and without any recourse to the dynamic programming principle. By analyzing the properties of the free boundary problem, we provide an explicit characterization of model parameters for which the value function is finite. Furthermore, we prove that the value function, as well as the slopes of the lines demarcating the no-trading region, can be expanded as a series of integer powers of [lambda superscript 1/3]. The coefficients of arbitrary order in this expansion can be computed. / text

Not available / text

I develop a numerical method that combines functional approximations and dynamic programming to solve high-dimensional discrete-time stochastic control problems under general constraints. The method relies on three building blocks: first, a quasi-random grid and the radial basis function method are used to discretize and interpolate the high-dimensional state space; second, to incorporate constraints, the method of Lagrange multipliers is applied to obtain the first order optimality conditions; third, the conditional expectation of the value function is approximated by a second order polynomial basis, estimated using ordinary least squares regressions. To reduce the approximation error, I introduce the test region iterative contraction (TRIC) method to shrink the approximation region around the optimal solution. I apply the method to two Finance applications: a) dynamic portfolio choice with constraints, a continuous control problem; b) dynamic portfolio choice with capital gain taxation, a high-dimensional singular control problem. / text

Advances in machine autonomy hold great promise in advancing technology, economic markets, and general societal well-being. For example, the progression of unmanned air systems (UAS) research has demonstrated the effectiveness and reliability of these autonomous systems in performing complex tasks. UAS have shown to not only outperformed human pilots in some tasks, but have also made novel applications not possible for human pilots practical. Nevertheless, human pilots are still favored when performing specific challenging tasks. For example, transportation of suspended (sometimes called slung or sling) loads requires highly skilled pilots and has only been performed by UAS in highly controlled environments. The presented work begins to bridge this autonomy gap by proposing a trajectory optimization framework for operations involving autonomous rotorcraft with suspended loads. The framework generates optimized vehicle trajectories that are used by existing guidance, navigation, and control systems and estimates the state of the non-instrumented load using a downward facing camera. Data collected from several simulation studies and a flight test demonstrates the proposed framework is able to produce effective guidance during autonomous suspended load operations. In addition, variational integrators are extensively studied in this dissertation. The derivation of a stochastic variational integrator is presented. It is shown that the presented stochastic variational integrator significantly improves the performance of the stochastic differential dynamical programming and the extended Kalman filter algorithms. A variational integrator for the propagation of polynomial chaos expansion coefficients is also presented. As a result, the expectation and variance of the trajectory of an uncertain system can be accurately predicted.

In the near future, demand for Heterogeneous Wireless Networking (HWN) is expected to increase. HWNs are formed by integration of different communication technologies, for example the integration of wireless LAN and cellular networks, to support mobile users. QoS provisioning in these networks is a challenging issue given the diversity in wireless technologies and the existence of mobile users with different communication requirements. In this thesis, we consider optimal resource planning and dynamic resource management for HWNs. In the first part of this thesis, we examine the optimal deployment of such networks. We propose a mobility-aware network planning optimization in which the objective is to minimize the rate of upward vertical handovers while maximizing the total number of users accommodated by the network. The optimal placement of Access Points (AP) with respect to these two objectives is formulated as an integer programming problem. Our results show that considering the mobility pattern in the planning phase of network deployment can significantly improve infrastructure performance. In the second part, we investigate optimal admission control policies employed in maintaining QoS in HWNs. Here we consider two cases: integration of cellular overlay with a single WLAN AP, and integration with a WLAN mesh network. A decision theoretic framework for the problem is derived using a dynamic programming formulation. In the case of single WLAN AP and cellular overlay, we prove that for this two-tier wireless network architecture, the optimal policy has a two-dimensional threshold structure. Furthermore, this structural result is used to design two computationally efficient algorithms, Structured Value Iteration and Structured Update Value Iteration. These algorithms can be used to determine the optimal policy in terms of thresholds. Although the first one is closer in its operation to the conventional Value Iteration algorithm, the second one has a significantly lower complexity. In the second case where the underlay is a complex WLAN mesh network, we develop a Partially Observable Markov-Modulated Poisson Process (PO-MMPP) traffic model to characterize the overflow traffic from the underlaying mesh to the overlay. This model captures the burstiness of the overflow traffic under the imperfect observability of the mesh network states. Then, by modeling the overlay network as a controlled PO-MMPP/M/C/C queueing system and obtaining structured decision theoretic results, it is shown that the optimal control policy for this class of HWNs can be characterized as monotonic \emph{threshold curves}. Moreover, these results are used to design a computationally efficient algorithm to determine the optimal policy in terms of thresholds. Extensive numerical observations suggest that, in both cases and for all practical parameter sets, the algorithms converge to the overall optimal policy. Additionally, numerical results show that the proposed algorithms are efficient in terms of time-complexity and in achieving optimal performance by significantly reducing the probability of dropped and blocked calls.

In the near future, demand for Heterogeneous Wireless Networking (HWN) is expected to increase. HWNs are formed by integration of different communication technologies, for example the integration of wireless LAN and cellular networks, to support mobile users. QoS provisioning in these networks is a challenging issue given the diversity in wireless technologies and the existence of mobile users with different communication requirements. In this thesis, we consider optimal resource planning and dynamic resource management for HWNs. In the first part of this thesis, we examine the optimal deployment of such networks. We propose a mobility-aware network planning optimization in which the objective is to minimize the rate of upward vertical handovers while maximizing the total number of users accommodated by the network. The optimal placement of Access Points (AP) with respect to these two objectives is formulated as an integer programming problem. Our results show that considering the mobility pattern in the planning phase of network deployment can significantly improve infrastructure performance. In the second part, we investigate optimal admission control policies employed in maintaining QoS in HWNs. Here we consider two cases: integration of cellular overlay with a single WLAN AP, and integration with a WLAN mesh network. A decision theoretic framework for the problem is derived using a dynamic programming formulation. In the case of single WLAN AP and cellular overlay, we prove that for this two-tier wireless network architecture, the optimal policy has a two-dimensional threshold structure. Furthermore, this structural result is used to design two computationally efficient algorithms, Structured Value Iteration and Structured Update Value Iteration. These algorithms can be used to determine the optimal policy in terms of thresholds. Although the first one is closer in its operation to the conventional Value Iteration algorithm, the second one has a significantly lower complexity. In the second case where the underlay is a complex WLAN mesh network, we develop a Partially Observable Markov-Modulated Poisson Process (PO-MMPP) traffic model to characterize the overflow traffic from the underlaying mesh to the overlay. This model captures the burstiness of the overflow traffic under the imperfect observability of the mesh network states. Then, by modeling the overlay network as a controlled PO-MMPP/M/C/C queueing system and obtaining structured decision theoretic results, it is shown that the optimal control policy for this class of HWNs can be characterized as monotonic \emph{threshold curves}. Moreover, these results are used to design a computationally efficient algorithm to determine the optimal policy in terms of thresholds. Extensive numerical observations suggest that, in both cases and for all practical parameter sets, the algorithms converge to the overall optimal policy. Additionally, numerical results show that the proposed algorithms are efficient in terms of time-complexity and in achieving optimal performance by significantly reducing the probability of dropped and blocked calls.

>Magister Scientiae - MSc / The industry of pension funds has become one of the drivers of today’s economic activity by its important volume of contribution in the financial market and by creating wealth. The increasing importance that pension funds have acquired in today’s economy and financial market, raises special attention from investors, financial actors and pundits in the sector. Regarding this economic weight of pension funds, a thorough analysis of the performance of different pension funds plans in order to optimise benefits need to be undertaken. The research explores criteria and invariants that make it possible to compare the performance of different pension fund products. Pension fund companies currently do measure their performances with those of others. Likewise, the individual investing in a pension plan compares different products available in the market. There exist different ways of measuring the performance of a pension fund according to their different schemes. Generally, there exist two main pension funds plans. The defined benefit (DB) pension funds plan which is mostly preferred by pension members due to his ability to hold the risk to the pension fund manager. The defined contributions (DC) pension fund plan on the other hand, is more popularly preferred by the pension fund managers due to its ability to transfer the risk to the pension fund members. One of the reasons that motivate pension fund members’ choices of entering into a certain programme is that their expectations of maintaining their living lifestyle after retirement are met by the pension fund strategies. This dissertation investigates the various properties and characteristics of the defined contribution pension fund plan with a minimum guarantee and benchmark in order to mitigate the risk that pension fund members are subject to. For the pension fund manager the aim is to find the optimal asset allocation strategy which optimises its retribution which is in fact a part of the surplus (the difference between the pension fund value and the guarantee) (2004) [19] and to analyse the effect of sharing between the contributor and the pension fund. From the pension fund members’ perspective it is to define a optimal guarantee as a solution to the contributor’s optimisation programme. In particular, we consider a case of a pension fund company which invests in a bond, stocks and a money market account. The uncertainty in the financial market is driven by Brownian motions. Numerical simulations were performed to compare the different models.

The first part of this research program concerns the development of customized and easily implementable Probably Approximately Correct (PAC)-learning algorithms for episodic tasks over acyclic state spaces. The defining characteristic of our algorithms is that they take explicitly into consideration the acyclic structure of the underlying state space and the episodic nature of the considered learning task. The first of the above two attributes enables a very straightforward and efficient resolution of the ``exploration vs exploitation' dilemma, while the second provides a natural regenerating mechanism that is instrumental in the dynamics of our algorithms. Some additional characteristics that distinguish our algorithms from those developed in the past literature are (i) their direct nature, that eliminates the need of a complete specification of the underlying MDP model and reduces their execution to a very simple computation, and (ii) the unique emphasis that they place in the efficient implementation of the sampling process that is defined by their PAC property. More specifically, the aforementioned PAC-learning algorithms complete their learning task by implementing a systematic episodic sampling schedule on the underlying acyclic state space. This sampling schedule combined with the stochastic nature of the transitions taking place, define the need for efficient routing policies that will help the algorithms complete their exploration program while minimizing, in expectation, the number of executed episodes. The design of an optimal policy that will satisfy a specified pattern of arc visitation requirements in an acyclic stochastic graph, while minimizing the expected number of required episodes, is a challenging problem, even under the assumption that all the branching probabilities involved are known a priori. Hence, the sampling process that takes place in the proposed PAC-learning algorithms gives rise to a novel, very interesting stochastic control/scheduling problem, that is characterized as the problem of the Optimal Node Visitation (ONV) in acyclic stochastic digraphs. The second part of the work presented herein seeks the systematic modelling and analysis of the ONV problem. The last part of this research program explores the computational merits obtained by heuristical implementations that result from the integration of the ONV problem developments into the PAC-algorithms developed in the first part of this work. We study, through numerical experimentation, the relative performance of these resulting heuristical implementations in comparison to (i) the initial version of the PAC-learning algorithms, presented in the first part of the research program, and (ii) standard Q-learning algorithm variations provided in the RL literature. The work presented in this last part reinforces and confirms the driving assumption of this research, i.e., that one can design customized RL algorithms of enhanced performance if the underlying problem structure is taken into account.

Thesis (MSc)--Stellenbosch University, 2011. / ENGLISH ABSTRACT: We consider the utility portfolio optimization problem of an investor whose activities are influenced by an exogenous financial risk (like bad weather or energy shortage) in an incomplete financial market. We work with a fairly general non-Markovian model, allowing stochastic correlations between the underlying assets. This important problem in finance and insurance is tackled by means of backward stochastic differential equations (BSDEs), which have been shown to be powerful tools in stochastic control. To lay stress on the importance and the omnipresence of BSDEs in stochastic control, we present three methods to transform the control problem into a BSDEs. Namely, the martingale optimality principle introduced by Davis, the martingale representation and a method based on Itô-Ventzell’s formula. These approaches enable us to work with portfolio constraints described by closed, not necessarily convex sets and to get around the classical duality theory of convex analysis. The solution of the optimization problem can then be simply read from the solution of the BSDE. An interesting feature of each of the different approaches is that the generator of the BSDE characterizing the control problem has a quadratic growth and depends on the form of the set of constraints. We review some recent advances on the theory of quadratic BSDEs and its applications. There is no general existence result for multidimensional quadratic BSDEs. In the one-dimensional case, existence and uniqueness strongly depend on the form of the terminal condition. Other topics of investigation are measure solutions of BSDEs, notably measure solutions of BSDE with jumps and numerical approximations. We extend the equivalence result of Ankirchner et al. (2009) between existence of classical solutions and existence of measure solutions to the case of BSDEs driven by a Poisson process with a bounded terminal condition. We obtain a numerical scheme to approximate measure solutions. In fact, the existing self-contained construction of measure solutions gives rise to a numerical scheme for some classes of Lipschitz BSDEs. Two numerical schemes for quadratic BSDEs introduced in Imkeller et al. (2010) and based, respectively, on the Cole-Hopf transformation and the truncation procedure are implemented and the results are compared. Keywords: BSDE, quadratic growth, measure solutions, martingale theory, numerical scheme, indifference pricing and hedging, non-tradable underlying, defaultable claim, utility maximization. / AFRIKAANSE OPSOMMING: Ons beskou die nuts portefeulje optimalisering probleem van ’n belegger wat se aktiwiteite beïnvloed word deur ’n eksterne finansiele risiko (soos onweer of ’n energie tekort) in ’n onvolledige finansiële mark. Ons werk met ’n redelik algemene nie-Markoviaanse model, wat stogastiese korrelasies tussen die onderliggende bates toelaat. Hierdie belangrike probleem in finansies en versekering is aangepak deur middel van terugwaartse stogastiese differensiaalvergelykings (TSDEs), wat blyk om ’n onderskeidende metode in stogastiese beheer te wees. Om klem te lê op die belangrikheid en alomteenwoordigheid van TSDEs in stogastiese beheer, bespreek ons drie metodes om die beheer probleem te transformeer na ’n TSDE. Naamlik, die martingale optimaliteits beginsel van Davis, die martingale voorstelling en ’n metode wat gebaseer is op ’n formule van Itô-Ventzell. Hierdie benaderings stel ons in staat om te werk met portefeulje beperkinge wat beskryf word deur geslote, nie noodwendig konvekse versamelings, en die klassieke dualiteit teorie van konvekse analise te oorkom. Die oplossing van die optimaliserings probleem kan dan bloot afgelees word van die oplossing van die TSDE. ’n Interessante kenmerk van elkeen van die verskillende benaderings is dat die voortbringer van die TSDE wat die beheer probleem beshryf, kwadratiese groei en afhanglik is van die vorm van die versameling beperkings. Ons herlei ’n paar onlangse vooruitgange in die teorie van kwadratiese TSDEs en gepaartgaande toepassings. Daar is geen algemene bestaanstelling vir multidimensionele kwadratiese TSDEs nie. In die een-dimensionele geval is bestaan ââen uniekheid sterk afhanklik van die vorm van die terminale voorwaardes. Ander ondersoek onderwerpe is maatoplossings van TSDEs, veral maatoplossings van TSDEs met spronge en numeriese benaderings. Ons brei uit op die ekwivalensie resultate van Ankirchner et al. (2009) tussen die bestaan van klassieke oplossings en die bestaan van maatoplossings vir die geval van TSDEs wat gedryf word deur ’n Poisson proses met begrensde terminale voorwaardes. Ons verkry ’n numeriese skema om oplossings te benader. Trouens, die bestaande self-vervatte konstruksie van maatoplossings gee aanleiding tot ’n numeriese skema vir sekere klasse van Lipschitz TSDEs. Twee numeriese skemas vir kwadratiese TSDEs, bekendgestel in Imkeller et al. (2010), en gebaseer is, onderskeidelik, op die Cole-Hopf transformasie en die afknot proses is geïmplementeer en die resultate word vergelyk.

This monograph addresses control and filtering problems for systems with sudden changes in their behavior and whose changes are detected and estimated by an imperfect detector. More precisely it considers continuous-timeMarkov Jump Linear Systems (MJLS) where the current mode of operation is estimated by a detector. This detector is assumed to be imperfect in the sense that it is possible that the detected mode of operation diverges from the real mode of operation. Furthermore the probabilities for these detections are considered to be known. It is assumed that the detector has its own dynamic, which means that the detected mode of information can change independently from the real mode of operation. The novelty of this approach lies in how uncertainties are modeled. A Hidden Markov Model (HMM) is used to model the uncertainties introduced by the detector. For these systems the following problems are addressed: i) Stochastic Stabilizability in mean-square sense, ii) H2 control, iii) H? control and iv) the H? filtering problem. Solutions based on Linear Matrix Inequalities (LMI) are developed for each of these problems. In case of the H2 control problem, the solutionminimizes an upper bound for the H2 norm of the closed-loop control system. For the H? control problem a solution is presented that minimizes an upper bound for the H? norm of the closed-loop control system. In the case of the H? filtering, the solution presented minimizes the H? norm of a system representing the estimation error. The solutions for the control problems are illustrated using a numerical example modeling a simple two-tank process. / Esta monografia aborda problemas de controle e filtragem em sistemas com saltos espontâneos que alteram seu comportamento e cujas mudanças são detectadas e estimadas por um detector imperfeito. Mais precisamente, consideramos sistemas lineares cujos saltos podem ser modelados usando um processo markoviano (Markov Jump Linear Systems) e cujo modo de operação corrente é estimado por um detector. O detector é considerado imperfeito tendo em vista a possibilidade de divergência entre o modo real de operação e o modo de operação detectado. Ademais, as probabilidades das deteccões são consideradas conhecidas. Assumimos que o detector possui uma dinâmica própria, o que significa que o modo de operação detectado pode mudar independentemente do modo real de operação. A novidade dessa abordagem está na modelagem das incertezas. Um processo oculto de Markov (HMM) é usado para modelar as incertezas introduzidas pelo detector. Para esses sistemas, os seguintes problemas são abordados: i) estabilidade quadrática ii) controle H2, iii) controle H? e iv) o problema da filtragem H?. Soluções baseadas em Desigualdades de Matriciais Lineares (LMI) são desenvolvidas para cada um desses problemas. No caso do problema de controle H2, a solução minimiza um limite superior para a norma H2 do sistema de controle em malha fechada. Para o problema H? -controle é apresentada uma solução que minimiza um limite superior para a norma H? do sistema de controle em malha fechada. No caso da filtragem H?, a solução apresentada minimiza a norma H? de um sistema que representa o erro de estimativa. As soluções para os problemas de controle são ilustradas usando um exemplo numérico que modela um processo simples de dois tanques.