Stochastic control problems that arise in sequential decision making applications typically assume that information used for decision-making is obtained according to a predetermined sampling schedule. In many real applications however, there is a high sampling cost associated with collecting such data. It is therefore of equal importance to determine when information should be collected as it is to decide how this information should be utilized for optimal decision-making. This type of joint optimization has been a long-standing problem in the operations research literature, and very few results regarding the structure of the optimal sampling and control policy have been published. In this thesis, the joint optimization of sampling and control is studied in the context of maintenance optimization. New theoretical results characterizing the structure of the optimal policy are established, which have practical interpretation and give new insight into the value of condition-based maintenance programs in life-cycle asset management. Applications in other areas such as healthcare decision-making and statistical process control are discussed. Statistical parameter estimation results are also developed with illustrative real-world numerical examples.

Guaranteed Minimum Withdrawal Benefits(GMWB) have become popular riders on variable annuities. The pricing of a GMWB contract was originally formulated as a singular stochastic control problem which results in a Hamilton Jacobi Bellman (HJB) Variational Inequality (VI). A penalty method method can then be used to solve the HJB VI. We present a rigorous proof of convergence of the penalty method to the viscosity solution of the HJB VI assuming the underlying asset follows a Geometric Brownian Motion. A direct control method is an alternative formulation for the HJB VI. We also extend the HJB VI to the case of where the underlying asset follows a Poisson jump diffusion. The HJB VI is normally solved numerically by an implicit method, which gives rise to highly nonlinear discretized algebraic equations. The classic policy iteration approach works well for the Geometric Brownian Motion case. However it is not efficient in some circumstances such as when the underlying asset follows a Poisson jump diffusion process. We develop a combined fixed point policy iteration scheme which significantly increases the efficiency of solving the discretized equations. Sufficient conditions to ensure the convergence of the combined fixed point policy iteration scheme are derived both for the penalty method and direct control method. The GMWB formulated as a singular control problem has a special structure which results in a block matrix fixed point policy iteration converging about one order of magnitude faster than a full matrix fixed point policy iteration. Sufficient conditions for convergence of the block matrix fixed point policy iteration are derived. Estimates for bounds on the penalty parameter (penalty method) and scaling parameter (direct control method) are obtained so that convergence of the iteration can be expected in the presence of round-off error.

Diss. (sammanfattning) Uppsala : Univ., 2003. / Härtill 6 uppsatser.

Thesis (Ph.D.)--University of Adelaide, Dept. of Electrical and Electronic Engineering, 2003. / "January 2003" Includes bibliographical references (leaves 120-129).

Made available in DSpace on 2016-12-12T20:27:38Z (GMT). No. of bitstreams: 1 Aldo Zanella Junior.pdf: 3746857 bytes, checksum: 7ff548689a89fd8090402ad4891a23c1 (MD5) Previous issue date: 2015-07-09 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / This work introduces the report of performed studies in order to evaluate the applicability of generalized predictive control (GPC) to several plants. The main goal is to analyze the GPC performance in processes with different features, analyzing the influence of its tuning parameters. The study is justified by the fact that GPC presents itself as a generalized solution for several classes of processes, which are becoming increasingly complex and demanding for traditional controllers to handle. For the purpose to prove this proposal of GPC, it was performed several tests with plants of different orders and response characteristics, real and simulated, varying controller tuning parameters and measuring some quality indices. It was evaluated the influence of tuning parameters and it was made a report of conclusions that was reached. Through obtained results, it is shown that GPC satisfies the proposal and presents favorable results. / Esta dissertação traz o relato do estudo realizado a fim de avaliar a aplicabilidade do controlador preditivo generalizado (GPC) em plantas diversas. O objetivo principal é analisar o desempenho do GPC em processos com diferentes características, analisando a influência dos seus parâmetros de sintonia. O estudo se justifica pelo fato de que o GPC apresenta-se como uma solução generalizada para diversos tipos de processos, os quais estão se tornando cada vez mais complexos e com maiores exigências para o controlador. A fim de comprovar essa proposta do GPC, realizou-se inúmeros ensaios com plantas com respostas e ordem diferentes, reais e simuladas, variando-se os parâmetros de sintonia do controlador e medindo-se alguns parâmetros de qualidade. Avaliou-se a influência dos parâmetros de sintonia e fez-se um relato das conclusões a que se chegou. Através dos resultados obtidos, mostra-se que o GPC corresponde ao que se propõe para as plantas testadas e apresenta resultados favoráveis.

Constraint handling is difficult in model predictive control (MPC) of linear differential inclusions (LDIs) and linear parameter varying (LPV) systems. The designer is faced with a choice of using conservative bounds that may give poor performance, or accurate ones that require heavy online computation. This thesis presents a framework to achieve a more flexible trade-off between these two extremes by using a state tube, a sequence of parametrised polyhedra that is guaranteed to contain the future state. To define controllers using a tube, one must ensure that the polyhedra are a sub-set of the region defined by constraints. Necessary and sufficient conditions for these subset relations follow from duality theory, and it is possible to apply these conditions to constrain predicted system states and inputs with only a little conservatism. This leads to a general method of MPC design for uncertain-parameter systems. The resulting controllers have strong theoretical properties, can be implemented using standard algorithms and outperform existing techniques. Crucially, the online optimisation used in the controller is a convex problem with a number of constraints and variables that increases only linearly with the length of the prediction horizon. This holds true for both LDI and LPV systems. For the latter it is possible to optimise over a class of gain-scheduled control policies to improve performance, with a similar linear increase in problem size. The framework extends to stochastic LDIs with chance constraints, for which there are efficient suboptimal methods using online sampling. Sample approximations of chance constraint-admissible sets are generally not positively invariant, which motivates the novel concept of ‘sample-admissible' sets with this property to ensure recursive feasibility when using sampling methods. The thesis concludes by introducing a simple, convex alternative to chance-constrained MPC that applies a robust bound to the time average of constraint violations in closed-loop.

O problema do desenvolvimento do \"Software\" para a pilotagem automática de embarcações é tratado através da aplicação de conceitos de controle estocástico. O movimento da embarcação é descrito de forma aproximada através do modelo clássico das derivadas hidrodinâmicas. A partir do Princípio da Separação desenvolve-se um procedimento sequencial em que os problemas de estimação do estado e de controle são tratados concomitantemente com as identificação de efeitos não modelados da dinâmica adotada para o piloto automático. A estimação de estados realiza-se através do filtro estendido de Kalman, que opera sobre informações de posição e velocidade da embarcação. O controlador, de natureza sequencial, atua sobre a embarcação em tempo discreto, sendo o leme o único elemento de controle. O piloto automático proposto foi testado através de simulação digital e alguns resultados obtidos são apresentados e discutidos. / The software development problem for the ship automatic steering is considered through the application of stochastic control concepts. Ship motion is described in an approximate way by the classical hydrodynamic derivatives model. A sequential procedure based on the Separation Principle is developed, being state estimation and control problems handled simultaneously with the identification of unmodeled effects of automatic pilot ship dynamics. State estimates are provided by the extended Kalman filter by using ship position and velocity measurements. The rudder is the only control element for the sequential, discrete-time controller. Digital simulation is employed for testing of the proposed automatic pilot, and some results are presented and discussed.

Orientador: João Bosco Ribeiro do Val / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de Computação / Made available in DSpace on 2018-08-14T00:07:24Z (GMT). No. of bitstreams: 1 Calmon_AndreduPin_M.pdf: 862345 bytes, checksum: 122780715dca28ac7fa3199aa0586e7c (MD5) Previous issue date: 2009 / Resumo: Este trabalho apresenta a caracterização teórica e a estratégia de controle para sistemas estocásticos em tempo discreto onde a variação da ação de controle aumenta a incerteza sobre o estado (sistemas VCAI). Este tipo de sistema possui várias aplicações práticas, como em problemas de política monetária, medicina e, de forma geral, em problemas onde um modelo dinâmico completo do sistema é complexo demais para ser conhecido. Utilizando ferramentas da análise de funções não suaves, mostra-se para um sistema VCAI multidimensional que a convexidade é uma invariante da função valor da Programação Dinâmica quando o custo por estágio é convexo. Esta estratégia indica a existência de uma região no espaço de estados onde a ação ótima de controle é de não variação (denominada região de não-variação), estando de acordo com a natureza cautelosa do controle de sistemas subdeterminados. Adicionalmente, estudou-se algoritmos para a obtenção da política ótima de controle para sistemas VCAI, com ênfase no caso mono-entrada avaliado através de uma função custo quadrática. Finalmente, os resultados obtidos foram aplicados no problema da condução da política monetária pelo Banco Central. / Abstract: This dissertation presents a theoretical framework and the control strategy for discrete-time stochastic systems for which the control variations increase state uncertainty (CVIU systems). This type of system model can be useful in many practical situations, such as in monetary policy problems, medicine and biology, and, in general, in problems for which a complete dynamic model is too complex to be feasible. The optimal control strategy for a multidimensional CVIU system associated with a convex cost functional is devised using dynamic programming and tools from nonsmooth analysis. Furthermore, this strategy points to a region in the state space in which the optimal action is of no variation (the region of no variation), as expected from the cautionary nature of controlling underdetermined systems. Numerical strategies for obtaining the optimal policy in CVIU systems were developed, with focus on the single-input input case evaluated through a quadratic cost functional. These results are illustrated through a numerical example in economics. / Mestrado / Automação / Mestre em Engenharia Elétrica

Orientador: João Bosco Ribeiro do Val / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de Computação / Made available in DSpace on 2018-08-14T11:15:46Z (GMT). No. of bitstreams: 1 Pinheiro_NathalieCarvalho_M.pdf: 2872807 bytes, checksum: 929137a3cc76534c1832b041f3175ff3 (MD5) Previous issue date: 2009 / Resumo: Os elevados custos de produção na extração de petróleo marítimo são reduzidos com um sistema inovador que a Petrobras está desenvolvendo, denominado VASPS (do inglês, Vertical Annular Separation and Pumping System), capaz de separar gás e líquido ainda no assoalho oceânico. Esta dissertação trata do ajuste do nível de l'iquido no reservatório do VASPS, via controle impulsional estocástico, que se distancia bastante de um regulador convencional. A vazão de entrada flutua e o controle consiste em alterar a velocidade da bomba de saída do tanque, mantendo o nível numa faixa de operação, na qual ele varia livremente sem prejuízo. Todavia, é necessário de um lado observar o risco de operar próximo aos extremos de nível e de outro minimizar o número de intervenções na velocidade da bomba para prolongar sua vida útil. Optou-se por modelar o sistema por meio de um processo de difusão e formular por controle impulsional. Para a sua solução, o controle impulsional é convertido em uma sequência de problemas de parada ótima iterados, resolvidos utilizando-se a Discretização do Valor Médio, MVS (do inglês, Mean Value Scheme). Esta dissertação introduz o uso do controle impulsional nesta aplicação além da técnica citada para resolver problemas de parada ótima. / Abstract: The high costs in offshore oil production are reduced with the use of an innovative system which has been developed by Petrobras, the Brazilian oil company. Called VASPS (Vertical Annular Separation and Pumping System), it consists of an undersea gas/liquid separator. This work presents a strategy for the liquid level adjustment in the VASPS tank, which is subject to uncertain liquid inflow. This is far from a strict control regulator problem, since the liquid level may drifts freely inside an operation range. Although, in one hand, it is necessary to account for a risky operation near the limits, on the other hand, acting freely and continuously in the controlled pump may drastically shorten the lifetime of the equipment. To prevent premature worn with halt in the oil production, the control input variations should be meager. We propose a stochastic impulse control for varying the outflow pump speed. This formulation is transformed in a sequence of iterated optimal stopping problems, which results in a sequence of variational inequalities. We employ the numerical method called Mean Value Scheme (MVS) to solve this type of problem. This monograph introduces impulse control to the resevoir level adjustment of VASPS, together with the application of the MVS to its solution. / Mestrado / Automação / Mestre em Engenharia Elétrica

In this thesis, an optimal investment problem is studied for an investor who can only invest in a financial market modelled by an Itô-Lévy process; with one risk free (bond) and one risky (stock) investment possibility. We present the dynamic programming method and the associated Hamilton-Jacobi-Bellman (HJB) equation to explicitly solve this problem. It is shown that with purification and simplification to the standard jump diffusion process, closed form solutions for the optimal investment strategy and for the value function are attainable. It is also shown that, an explicit solution can be obtained via a finite training of a neural network using Stochastic gradient descent (SGD) for a specific case.