The scientific domain of this thesis is optimization under uncertainty for discrete event stochastic systems. In particular, this thesis focuses on the practical implementation of the Dynamic Programming (DP) methodology to discrete event stochastic systems. Unfortunately DP in its crude form suffers from three severe computational obstacles that make its imple-mentation to such systems an impossible task. This thesis addresses these obstacles by developing and executing practical Approximate Dynamic Programming (ADP) techniques. Specifically, for the purposes of this thesis we developed the following ADP techniques. The first one is inspired from the Reinforcement Learning (RL) literature and is termed as Real Time Approximate Dynamic Programming (RTADP). The RTADP algorithm is meant for active learning while operating the stochastic system. The basic idea is that the agent while constantly interacts with the uncertain environment accumulates experience, which enables him to react more optimal in future similar situations. While the second one is an off-line ADP procedure These ADP techniques are demonstrated on a variety of discrete event stochastic systems such as: i) a three stage queuing manufacturing network with recycle, ii) a supply chain of the light aromatics of a typical refinery, iii) several stochastic shortest path instances with a single starting and terminal state and iv) a general project portfolio management problem. Moreover, this work addresses, in a systematic way, the issue of multistage risk within the DP framework by exploring the usage of intra-period and inter-period risk sensitive utility functions. In this thesis we propose a special structure for an intra-period utility and compare the derived policies in several multistage instances.

Thesis (Ph.D.)--Mississippi State University. Department of Physics and Astronomy. / Title from title screen. Includes bibliographical references.

We study numerical methods for finding the maximal symmetric positive definite solution of the nonlinear matrix equation $X = Q + LX^{-1}L^T$, where Q is symmetric positive definite and L is nonsingular. Such equations arise for instance in the analysis of stationary Gaussian reciprocal processes over a finite interval. Its unique largest positive definite solution coincides with the unique positive definite solution of a related discrete-time algebraic Riccati equation (DARE). We discuss how to use the butterfly SZ algorithm to solve the DARE. This approach is compared to several fixed point type iterative methods suggested in the literature.

The numerical solution of Stein (aka discrete Lyapunov) equations is the primary step in Newton's method for the solution of discrete-time algebraic Riccati equations (DARE). Here we present a low-rank Smith method as well as a low-rank alternating-direction-implicit-iteration to compute low-rank approximations to solutions of Stein equations arising in this context. Numerical results are given to verify the efficiency and accuracy of the proposed algorithms.

The present work introduces a methodology to obtain a discrete time state space representation of an electrical network using the nodal [G] matrix of the Electromagnetic Transients Program (EMTP) solution. This is the first time the connection between the EMTP nodal analysis solution and a corresponding state-space formulation is presented. Compared to conventional state space solutions, the nodal EMTP solution is computationally much more efficient. Compared to the phasor solutions used in transient stability analysis, the proposed approach captures a much wider range of eigenvalues and system operating states. A fundamental advantage of extracting the system eigenvalues directly from the EMTP solution is the ability of the EMTP to follow the characteristics of nonlinearities. The system's trajectory can be accurately traced and the calculated eigenvalues and eigenvectors correctly represent the system's instantaneous dynamics. In addition, the algorithm can be used as a tool to identify network partitioning subsystems suitable for real-time hybrid power system simulator environments, including the implementation of multi-time scale solutions. The proposed technique can be implemented as an extension to any EMTP-based simulator. Within our UBC research group, it is aimed at extending the capabilities of our real-time PC-cluster Object Virtual Network Integrator (OVNI) simulator.

Consider two Markovian N-policy queueing models in discrete time, one with batch arrival, the other with batch service. In this paper the transient behaviour of both models is studied and the analogous continuous time results are achieved by a limiting process. The steady state solution for the model with batch arrival is derived. (author's abstract) / Series: Forschungsberichte / Institut für Statistik

This work is an extension of the classical Cox-Ross-Rubinstein discrete time market model in which only one risky asset is considered. We introduce another risky asset into the model. Moreover, the random structure of the asset price sequence is generated by bivariate finite state Markov chain. Then, the interest rate varies over time as it is the function of generating sequences. We discuss how the model can be adapted to the real data. Finally, we illustrate sample implementations to give a better idea about the use of the model.

This thesis investigates the frequency weighted balanced model reduction problem for linear time invariant systems. Both continuous and discrete time systems are considered, in one and two-dimensions. First the frequency weighted balanced model reduction problem is formulated, then a novel frequency weighted, balanced, model reduction method for continuous time systems is proposed. This method is based on the retention of frequency weighted Hankel singular values of the original system, and yields stable reduced order models even when two sided weightings are employed. An alternative frequency weighted balanced model reduction technique (applicable for controller reduction applications) is then developed. This is based on a parametrized combination of the frequency weighted partial fraction expansion technique with balanced truncation and the singular perturbation approximation techniques. This method yields stable models even when two sided weightings are employed. An a priori error bound for the model reduction method is derived. Lower frequency response errors and error bounds are obtained using free parameters and equivalent anti-stable weightings. Based on the same idea, a novel parameterized frequency weighted optimal Hankel norm model reduction method with a tighter a priori error bound is proposed. The proposed methods are extended to include discrete time systems. A frequency interval Gramians based stability preserving model reduction scheme with error bounds is also presented. In this case, frequency weights are not explicitly predefined. Discrete time system related results are also included. Several frequency weighted model reduction results for two-dimensional (2-D) systems are also proposed. The advantages of these schemes include error bounds, guaranteed stability and applicability to general stable (non-separable denominator) weighting functions. Finally, a novel 2-D identification based frequency weighted model reduction scheme is outlined. Numerically robust algorithms based on square root and balancing free techniques are proposed for frequency weighted balanced truncation techniques. Several practical examples are included to illustrate the effectiveness of the algorithms.

Este trabalho apresenta uma proposta de um software de simulação e diagnóstico de falhas aplicado a um sistema de automação. para realizar esta tarefa foi desenvolvido um estudo de caso que contem um problema de automação industrial. Deste estudo foi retirado um modelo, usado como base para criar do software de simulação que segue as especificações dos modelos, no qual é possível observar a evolução dos eventos através de sua linguagem e diagnosticar falhas, quando estas ocorrem em uma das máquinas da planta. O processo de diagnóstico de falhas tem como base as regras de diagnosticabilidade apresentadas na literatura. As máquinas, assim como suas restrições, são modeladas dentro do próprio software, onde são definidas as estruturas de cada automato assim como características como observabilidade, controlabilidade e a possibilidade de falhar. Os testes realizados no sistema apresentado mostraram que a linguagem de controle consegue, mesmo permitindo um nivel de liberdade ao sistema, identificar as falhas verificando somente para os eventos que ocorrem após a ocorrência da mesma. / This work we presents a proposal a software of simulation and fault diagnosis applied to an automation system. In that task was developed , a case study that contains a problem of industrial automation. The study was used as the source for creating the simulation that follows the specifications of software models, where you can observe the evolution of events through its language and diagnose faults when they occur in one of the machines of the plant. The process of fault diagnosis is based on the rules of diagnosticabilidades presented in the literature. The machines, as well as its restrictions are modeled within the software itself, where the structures of each automaton are defined the caracteristics as observability an the possibility of failure. The tests in the system showed that of the language used can allowing identify fails, checking only for events that occorem after the occurence the fails.

Este trabalho apresenta uma proposta de um software de simulação e diagnóstico de falhas aplicado a um sistema de automação. para realizar esta tarefa foi desenvolvido um estudo de caso que contem um problema de automação industrial. Deste estudo foi retirado um modelo, usado como base para criar do software de simulação que segue as especificações dos modelos, no qual é possível observar a evolução dos eventos através de sua linguagem e diagnosticar falhas, quando estas ocorrem em uma das máquinas da planta. O processo de diagnóstico de falhas tem como base as regras de diagnosticabilidade apresentadas na literatura. As máquinas, assim como suas restrições, são modeladas dentro do próprio software, onde são definidas as estruturas de cada automato assim como características como observabilidade, controlabilidade e a possibilidade de falhar. Os testes realizados no sistema apresentado mostraram que a linguagem de controle consegue, mesmo permitindo um nivel de liberdade ao sistema, identificar as falhas verificando somente para os eventos que ocorrem após a ocorrência da mesma. / This work we presents a proposal a software of simulation and fault diagnosis applied to an automation system. In that task was developed , a case study that contains a problem of industrial automation. The study was used as the source for creating the simulation that follows the specifications of software models, where you can observe the evolution of events through its language and diagnose faults when they occur in one of the machines of the plant. The process of fault diagnosis is based on the rules of diagnosticabilidades presented in the literature. The machines, as well as its restrictions are modeled within the software itself, where the structures of each automaton are defined the caracteristics as observability an the possibility of failure. The tests in the system showed that of the language used can allowing identify fails, checking only for events that occorem after the occurence the fails.