The optimal estimation theory of the Wiener-Kalman filter is extended to cover the situation in which the number of memory elements in the estimator is restricted. A method, based on the simultaneous diagonalisation of two symmetric positive definite matrices, is given which allows the weighted least square estimation error to be minimised. A control system design method is developed utilising this estimator, and this allows the dynamic controller in the feedback path to have a low order. A 12-order once-through boiler model is constructed and the performance of controllers of various orders generated by the design method is investigated. Little cost penalty is found even for the one-order controller when compared with the optimal Kalman filter system. Whereas in the Kalman filter all information from past observations is stored, the given method results in an estimate of the state variables which is a weighted sum of the selected information held in the storage elements. For the once-through boiler these weighting coefficients are found to be smooth functions of position, their form illustrating the implicit model reduction properties of the design method. Minimal-order estimators of the Luenberger type also generate low order controllers and the relation between the two design methods is examined. It is concluded that the design method developed in this thesis gives better plant estimates than the Luenberger system and, more fundamentally, allows a lower order control system to be constructed. Finally some possible extensions of the theory are indicated. An immediate application is to multivariable control systems, while the existence of a plant state estimate even in control systems of very low order allows a certain adaptive structure to be considered for systems with time-varying parameters.

Este trabalho consiste no estudo de métodos de otimização aplicados em um problema de controle para sistemas lineares com saltos markovianos (SLSM). SLSM formam uma importante classe de sistemas que têm sido muito úteis em aplicações envolvendo sistemas sujeitos a falhas e outras alterações abruptas de comportamento. Este estudo enfoca diferentes métodos para resolução deste problema. Comparamos o método variacional com o de Newton, sob o ponto de vista do número de problemas resolvidos e pelo nível de sub-otimalidade obtido (relação entre os custos obtidos por estes métodos). Também propomos um novo método, o qual pode ser inicializado com soluções de equações de Riccati acopladas, e o comparamos com o método variacional. Além disso, para a comparação dos métodos, propomos um algoritmo que gerou dez mil exemplos / This work addresses optimizations methods applied to a control problem for linear systems with markovian jumps, which form an important class of systems that have been very useful in applications involving systems subject to failures and other abrupt changes. This study focuses on different methods for solving this problem. We compare the variational approach with the Newton method, in terms of the number of solved problems and the level of sub-optimality (ratio between the costs obtained by these approaches). We also propose a new method, which can be initialized with solutions of coupled Riccati equations, and we compare it with the variational approach. We have proposed an algorithm for creating ten thousand examples for the comparisons

The scope of the thesis is to estimate the parameters of continuous-time models used within control and communication from sampled data with high accuracy and in a computationally efficient way.In the thesis, continuous-time models of systems controlled in a networked environment, errors-in-variables systems, stochastic closed-loop systems, and wireless channels are considered. The parameters of a transfer function based model for the process in a networked control system are estimated by a covariance function based approach relying upon the second order statistical properties of input and output signals. Some other approaches for estimating the parameters of continuous-time models for processes in networked environments are also considered. The multiple input multiple output errors-in-variables problem is solved by means of a covariance matching algorithm. An analysis of a covariance matching method for single input single output errors-in-variables system identification is also presented. The parameters of continuous-time autoregressive exogenous models are estimated from closed-loop filtered data, where the controllers in the closed-loop are of proportional and proportional integral type, and where the closed-loop also contains a time-delay. A stochastic differential equation is derived for Jakes's wireless channel model, describing the dynamics of a scattered electric field with the moving receiver incorporating a Doppler shift. / <p>The thesis consists of five main parts, where the first part is an introduction- Parts II-IV are based on the following articles:</p><p><strong>Part II</strong> - Networked Control Systems</p><p>1. Y. Irshad, M. Mossberg and T. Söderström. <em>System identification in a networkedenvironment using second order statistical properties</em>.</p><p>A versionwithout all appendices is published as Y. Irshad, M. Mossberg and T. Söderström. <em>System identification in a networked environment using second order statistical properties</em>. Automatica, 49(2), pages 652–659, 2013.</p><p>Some preliminary results are also published as M. Mossberg, Y. Irshad and T. Söderström. <em>A covariance function based approachto networked system identification.</em> In Proc. 2nd IFAC Workshop on Distributed Estimation and Control in Networked Systems, pages 127–132, Annecy,France, September 13–14, 2010</p><p>2. Y. Irshad and M. Mossberg. <em>Some parameters estimation methods applied tonetworked control systems</em>.A journal submission is made. Some preliminary results are published as Y. Irshad and M. Mossberg.<em> A comparison of estimation concepts applied to networked control systems</em>. In Proc. 19th Int. Conf. on Systems, Signals andImage Processing, pages 120–123, Vienna, Austria, April 11–13, 2012.</p><p><strong>Part III</strong> - Errors-in-variables Identification</p><p>3. Y. Irshad and M. Mossberg. <em>Continuous-time covariance matching for MIMOEIV system identification</em>. A journal submission is made.</p><p>4. T. Söderström, Y. Irshad, M. Mossberg and W. X. Zheng. <em>On the accuracy of acovariance matching method for continuous-time EIV identification. </em>Provisionally accepted for publication in Automatica.</p><p>Some preliminary results are published as T. Söderström, Y. Irshad, M. Mossberg, and W. X. Zheng. <em>Accuracy analysis of a covariance matching method for continuous-time errors-in-variables system identification</em>. In Proc. 16th IFAC Symp. System Identification, pages 1383–1388, Brussels, Belgium, July 11–13, 2012.</p><p><strong>Part IV</strong> - Wireless Channel Modeling</p><p>5. Y. Irshad and M. Mossberg.<em> Wireless channel modeling based on stochasticdifferential equations .</em>Some results are published as M. Mossberg and Y. Irshad.<em> A stochastic differential equation forwireless channelsbased on Jakes’s model with time-varying phases,</em> In Proc. 13th IEEEDigitalSignal Processing Workshop, pages 602–605, Marco Island, FL, January4–7, 2009.</p><p><strong>Part V</strong> - Closed-loop Identification</p><p>6. Y. Irshad and M. Mossberg. Closed-loop identification of P- and PI-controlledtime-delayed stochastic systems.Some results are published as M. Mossberg and Y. Irshad. <em>Closed-loop identific ation of stochastic models from filtered data</em>, In Proc. IEEE Multi-conference on Systems and Control,San Antonio, TX, September 3–5, 2008</p>

The scientific domain of this thesis is optimization under uncertainty for discrete event stochastic systems. In particular, this thesis focuses on the practical implementation of the Dynamic Programming (DP) methodology to discrete event stochastic systems. Unfortunately DP in its crude form suffers from three severe computational obstacles that make its imple-mentation to such systems an impossible task. This thesis addresses these obstacles by developing and executing practical Approximate Dynamic Programming (ADP) techniques. Specifically, for the purposes of this thesis we developed the following ADP techniques. The first one is inspired from the Reinforcement Learning (RL) literature and is termed as Real Time Approximate Dynamic Programming (RTADP). The RTADP algorithm is meant for active learning while operating the stochastic system. The basic idea is that the agent while constantly interacts with the uncertain environment accumulates experience, which enables him to react more optimal in future similar situations. While the second one is an off-line ADP procedure These ADP techniques are demonstrated on a variety of discrete event stochastic systems such as: i) a three stage queuing manufacturing network with recycle, ii) a supply chain of the light aromatics of a typical refinery, iii) several stochastic shortest path instances with a single starting and terminal state and iv) a general project portfolio management problem. Moreover, this work addresses, in a systematic way, the issue of multistage risk within the DP framework by exploring the usage of intra-period and inter-period risk sensitive utility functions. In this thesis we propose a special structure for an intra-period utility and compare the derived policies in several multistage instances.

Advanced process control solutions are oftentimes inadequate in their handling of uncertainty and disturbances. The main contribution of this work is to address this issue by providing solutions of immediate relevance to process control practitioners. To meet increasing performance demands, this work considers a Hidden Markov Model-based framework for describing non-stationary disturbance signals of practical interest such as intermittent drifts and abrupt jumps. The result is a more sophisticated model used by the state estimator for jump systems. At the expense of slightly higher computational costs (due to the state estimator), the proposed HMM disturbance model provides better tracking compared to a state estimator based on the commonly employed (in process control) integrated white noise disturbance model. Better tracking performance translates to superior closed loop performance without any redesign of the controller, through the typical assumption of separation and certainty equivalence. As a result, this provides a tool that can be readily adopted by process control practitioners. In line with this, the second aim is to develop approximate dynamic programming techniques for the rigorous control of nonlinear stochastic jump systems. The contribution is the creation of a framework that treats uncertainty in a systematic manner whilst leveraging existing off-the-shelf optimization solvers commonly employed by control practitioners.

We study two classes of stochastic systems, the limited processor sharing system and the multi-server system. They share the common feature that multiple jobs/customers are being processed simultaneously, which makes the study of them intrinsically difficult. In the limited processor sharing system, a limited number of jobs can equally share a single server, and the excess ones wait in a first-in-first-out buffer. The model is mainly motivated by computer related applications, such as database servers and packet transmission over the Internet. This model is studied in the first part of the thesis. The multi-server queue is mainly motivated by call centers, where each customer is handled by an agent. The number of customers being served at any time is limited by number of agents employed. Customers who can not be served upon arrival wait in a first-in-first-out buffer. This model is studied in the second part of the thesis.

Dažnai realių stochastinių sistemų negalime aprašyti Markovo procesais, nes operacijų trukmės nėra pasiskirstę pagal eksponentinį dėsnį. Šiame darbe nagrinėjome sistemų aproksimavimo galimybes, taikant eksponentinių skirstinių mišinius ir sąsūkas. Skirstinių aproksimavimui taikėme Erlango mišinius ir Kokso skirstinį. Skirstinių aproksimavimą pritaikėme aptarnavimo sistemų M/G/1 ir G/M/1 tyrimui. Atlikti teoriniai skaičiavimai parodė, kad gaunamas aukštas aproksimavimo tikslumas. Aptarnavimo sistemų modeliavimui naudojome skaitmeninio Markovo procesų modeliavimo sistemą naudojant įvykių kalbą. Darbe sukurti metodai leidžia tiksliai apskaičiuoti sistemų charakteristikas, naudojant aproksimavimą eksponentiniais mišiniais ir sąsūkomis. Sukurta programinė įranga leidžia automatizuoti sistemų M/G/1 ir G/M/1 modeliavimą, naudojant aproksimavimą eksponentiniais mišiniais. Sistemos G/G/1 ( neištiriamos analiziniais metodais ) aproksimavimo rezultatai leidžia tikėtis, kad šiame darbe nagrinėjamas metodas gali būti naudojamas ir sudėtingų sistemų modeliavime. / Application of numerical methods with approximation allows to extend a class of systems represented by Markovian processes under investigation compared with analytical methods. In this paper we used approximation of positive distribution functions, using phase-type distributions: mixtures of Erlang distributions and Coxian distribution – both 2 and 3 moments-matching algorithms was used. Analysis of M/G/1 and G/M/1 queueing systems showed, that moment-based queueing approximation gives high accuracy. In purpose to compute characteristics of M/G/1 and G/M/1 systems described in an event-based language, algorithms and software was created. Comparison to simulation results shows, that event-based language enables to get more precise results. Analysis of G/G/1 systems showed, that moment-based approximation can be used to analyse difficult queueing systems.

Orientadores: João Bosco Ribeiro do Val, Eduardo Fontoura Costa / Tese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenahria Eletrica e de Computação / Made available in DSpace on 2018-08-13T22:22:47Z (GMT). No. of bitstreams: 1 Vargas_AlessandrodoNascimento_D.pdf: 1972819 bytes, checksum: 4746f70b31b4471193e27d462a933bc0 (MD5) Previous issue date: 2009 / Resumo: Esta monografia apresenta resultados de estabilidade e controle de sistemas estocásticos representados por operadores lineares com respeito ao estado e não-lineares em relação ao controle, quando avaliados no critério de custo médio a longo prazo (CMLP). A estrutura de controle n¿ao depende da historia do processo e pode ser usada, como caso particular, para representar diversos problemas de controle existentes na literatura. Em relação a estabilidade, mostra-se que o sistema estocástico e assintoticamente estável na media se o custo CMLP 'e finito e se as hipóteses de controlabilidade e observabilidade são validas. Para garantir a estabilidade uniforme do segundo momento do sistema, algumas condições adicionais são verificadas. Em relação ao controle, apresentam-se condições que asseguram a existência de política ótima estacionaria no problema CMLP para a classe de sistemas estudados. Uma aproximação é desenvolvida para se obter o mínimo CMLP, e esta aproximação é ilustrada numericamente no problema de regulação de sistemas lineares sujeitos a saltos markovianos, supondo que o controlador não possui acesso ao estado de Markov. Exemplos numéricos são empregados para ilustrar a teoria desenvolvida. / Abstract: This monograph presents results on stability and control of stochastic systems represented by linear operators with respect to the state which are non-linear with respect to the control. The control seeks to optimize a long run average cost (LRAC). The control structure does not depend on the past history of the process and it can be used, in particular, to represent a broad range of control problems that appears in the literature. Regarding the stability, it is shown that the stochastic system is asymptotically stable in the mean if the LRAC is finite and if controllability and observability assumptions are satisfied. To guarantee the uniform second moment stability, some additional conditions must be verified. With respect to the control, the main goal is to assure the existence of an optimal stationary policy for the LRAC problem within the class of systems considered, and some independent conditions are derived. An approximation for the minimum LRAC is obtained, and it is illustrated numerically for the regulator problem of Markov jump linear systems, under the assumption that the controller does not have access to the Markov state. Numerical examples illustrate the derived theory. / Doutorado / Automação / Doutor em Engenharia Elétrica

Orientador: Furio Damiani / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de Computação / Made available in DSpace on 2018-08-21T21:06:15Z (GMT). No. of bitstreams: 1 Feijao_PedroCipriano_M.pdf: 2409509 bytes, checksum: ef809fef536991065abf131c6a599340 (MD5) Previous issue date: 2004 / Resumo: Um levantamento detalhado dos trabalhos em composição computacional será apresentado. Um algoritmo de composição e performance em tempo real de solos de jazz foi criado, inspirado no estilo bebop. Técnicas utilizadas por músicos de bebop durante os seus improvisos foram implementadas usando-se várias regras. Um novo modelo estocástico, baseado no raciocínio de um músico durante o seu solo, escolhe a saída do algoritmo. O usuário pode avaliar a saída do algoritmo em tempo-real, e isto será utilizado em um processo de aprendizado supervisionado para modificar os parâmetros do modelo estocástico. Algumas amostras de saídas do algoritmo serão mostradas, junto com as conclusões / Abstract: An extensive review of the most prominent works in the field of computer music is presented. A real-time algorithm for composition and performance of jazz solos in the style of bebop was created. Techniques that bebop musicians use on their solos were implemented using several rules. A new stochastic model, inspired by the thinking of a musician during his solo chooses the algorithm output. The user evaluates the algorithm output in real-time, and this input is used in a supervised learning process to change the stochastic model parameters. Samplings of the algorithm output are shown, along with concluding remarks / Mestrado / Eletrônica, Microeletrônica e Optoeletrônica / Mestre em Engenharia Elétrica

Orientadores: Paulo Regis Caron Ruffino, Pedro Jose Catuogno / Dissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Matematica, Estatistica e Computação Cientifica / Made available in DSpace on 2018-08-04T04:01:14Z (GMT). No. of bitstreams: 1 Vieira_MarceloGoncalvesOliveira_M.pdf: 798971 bytes, checksum: f8f23c9cc6bf1a5acb8f987a59bb2b28 (MD5) Previous issue date: 2005 / Resumo: Este trabalho aborda a homotopia monotônica, uma variante apropriada de homotopia, de trajetórias de sistemas de controle. Primeiro é introduzido um conceito de regularidade para funções de controle e depois é considerada a definição de homotopia monotônica de trajetórias regulares de um sistema de controle 'sigma' evoluindo sobre uma variedade M. Em seguida são mostrados que o conjunto 'gama' ('sigma', x) de classes de homotopia monotônica das trajetórias regulares do sistema 'sigma' a partir de um estado fixo tem um estrutura de variedade diferenciável. Outro resultado importante é a caracterização para trajetórias monotonicamente homotópicas (contidas no conjunto dos pontos acessíveis a partir de x) via os levantamentos das mesmas à variedade 'gama' ('sigma', x). Finalmente, são feitas considerações sobre homotopia monotônica e trajetórias de um sistema estocástico / Abstract: This work accosts the monotonic homotopy, an appropriate variant of homotopy of trajectories of control systems. It is introduced a concept of regularity for control functions and it is considered the de¯nition of monotonic homotopy of regular trajectories of a given control system 'sigma' on a manifold M. Then it is shown that the set 'gama' ('sigma', x) of monotonic homotopy classes of regular trajectories of 'sigma' starting at a given fixed point x has a differentiable manifold structure with the same dimension of M. Another important result is the caracterization of monotonic homotopy of trajectories (on acessible points set starting at x) via the lifts of same trajectories on the manifold 'gama' ('sigma', x). Finally, we make same considerations about monotonic homotopy and trajectories of an stochastic system / Mestrado / Matematica / Mestre em Matemática