Closed-loop system identification and fault detection and isolation are the two fundamental building blocks of process monitoring. Efficient and accurate process monitoring increases plant availability and utilisation. This dissertation investigates a subspace system identification and fault detection methodology for the Benfield process, used by Sasol, Synfuels in Secunda, South Africa, to remove CO2 from CO2-rich tail gas. Subspace identification methods originated between system theory, geometry and numerical linear algebra which makes it a computationally efficient tool to estimate system parameters. Subspace identification methods are classified as Black-Box identification techniques, where it does not rely on a-priori process information and estimates the process model structure and order automatically. Typical subspace identification algorithms use non-parsimonious model formulation, with extra terms in the model that appear to be non-causal (stochastic noise components). These extra terms are included to conveniently perform subspace projection, but are the cause for inflated variance in the estimates, and partially responsible for the loss of closed-loop identifiably. The subspace identification methodology proposed in this dissertation incorporates two successive LQ decompositions to remove stochastic components and obtain state-space models of the plant respectively. The stability of the identified plant is further guaranteed by using the shift invariant property of the extended observability matrix by appending the shifted extended observability matrix by a block of zeros. It is shown that the spectral radius of the identified system matrices all lies within a unit boundary, when the system matrices are derived from the newly appended extended observability matrix. The proposed subspace identification methodology is validated and verified by re-identifying the Benfield process operating in closed-loop, with an RMPCT controller, using measured closed-loop process data. Models that have been identified from data measured from the Benfield process operating in closed-loop with an RMPCT controller produced validation data fits of 65% and higher. From residual analysis results, it was concluded that the proposed subspace identification method produce models that are accurate in predicting future outputs and represent a wide variety of process inputs. A parametric fault detection methodology is proposed that monitors the estimated system parameters as identified from the subspace identification methodology. The fault detection methodology is based on the monitoring of parameter discrepancies, where sporadic parameter deviations will be detected as faults. Extended Kalman filter theory is implemented to estimate system parameters, instead of system states, as new process data becomes readily available. The extended Kalman filter needs accurate initial parameter estimates and is thus periodically updated by the subspace identification methodology, as a new set of more accurate parameters have been identified. The proposed fault detection methodology is validated and verified by monitoring process behaviour of the Benfield process. Faults that were monitored for, and detected include foaming, flooding and sensor faults. Initial process parameters as identified from the subspace method can be tracked efficiently by using an extended Kalman filter. This enables the fault detection methodology to identify process parameter deviations, with a process parameter deviation sensitivity of 2% or higher. This means that a 2% parameter deviation will be detected which greatly enhances the fault detection efficiency and sensitivity. / Dissertation (MEng)--University of Pretoria, 2008. / Electrical, Electronic and Computer Engineering / unrestricted

Stream processing systems have been widely adopted in applications such as recommendation systems, anomaly detection, and system monitoring due to their real-time capabilities. Improving observability in stream processing systems can further expand their application scenarios, including the implementation of stateful serverless applications. Stateful serverless applications are an emerging model in serverless computing that focuses on addressing the challenges of state management, enabling developers to build distributed applications in a simpler way. One possible implementation of stateful serverless applications is based on stream processing engines. However, the current approaches for observability in stream processing engines suffer from issues such as efficiency, consistency, and functionality, resulting in limited practical use cases. To address these challenges, we propose Queryable Workflow, an extension to stream processing engines. This extension allows users to access or modify the state within stream processing engines with transactional semantics using a SQL interface, enabling use cases such as ad-hoc querying, serializable updates, or even stateful serverless applications. We implemented our system on stream processing engines such as Portals and Apache Flink, and evaluated their performance. The result showed that our system has achieved 4.33x throughput improvement and 30% latency reduction compared to a baseline implemented with Apache Flink and Apache Kafka. With hand-crafted optimizations, our system achieved to process over 29,000 queries per second with a 99th percentile latency of 8.58 ms under a single-threaded runtime. Our proposed system provides a viable option for implementing stateful serverless applications that require transactional guarantees, while also expanding the potential application scenarios for stream processing engines. / Strömbehandlingssystem har på grund av sina realtidsegenskaper fått stor spridning i tillämpningar som rekommendationssystem, anomalidetektering och systemövervakning. Förbättrad observerbarhet i stream processing-system kan ytterligare utöka deras tillämpningsscenarier, inklusive implementeringen av stateful serverless-applikationer. Stateful serverless-applikationer är en framväxande modell inom serverless computing som fokuserar på att hantera utmaningarna med tillståndshantering, vilket gör det möjligt för utvecklare att bygga distribuerade applikationer på ett enklare sätt. En möjlig implementering av stateful serverless-applikationer är baserad på stream processing-motorer. De nuvarande metoderna för observerbarhet i strömbehandlingsmotorer lider dock av problem som effektivitet, konsistens och funktionalitet, vilket resulterar i begränsade praktiska användningsfall. För att ta itu med dessa utmaningar föreslog vi Queryable Workflow, ett tillägg till stream processing-motorer. Med detta tillägg kan användare komma åt eller ändra tillståndet i strömbehandlingsmotorer med transaktionssemantik med hjälp av ett SQL-gränssnitt, vilket möjliggör användningsfall som ad hoc-förfrågningar, serialiserbara uppdateringar eller till och med serverlösa applikationer med tillstånd. Vi implementerade vårt system på stream processing-motorer som Portals och Apache Flink, och utvärderade deras prestanda. Resultatet visade att vårt system har förbättrat genomströmningen 4,33 gånger och minskat latensen med 30% jämfört med en baslinje som implementerats med Apache Flink och Apache Kafka. Med handgjorda optimeringar lyckades vårt system bearbeta över 29 000 frågor per sekund med en 99:e percentil latens på 8,58 ms under en enkeltrådad körtid. Vårt föreslagna system har gett ett hållbart alternativ för att implementera stateful serverless-applikationer som kräver transaktionsgarantier, samtidigt som det också utökat de potentiella applikationsscenarierna för stream processing-motorer.

In this dissertation we present advances in developing an autonomous air vehicle capable of navigating through unknown, indoor environments. The problem imposes stringent limits on the computational power available onboard the vehicle, but the environment necessitates using 3D sensors such as stereo or RGB-D cameras whose data requires significant processing. We address the problem by proposing and developing key elements of a relative navigation scheme that moves as many processing tasks as possible out of the time-critical functions needed to maintain flight. We present in Chapter 2 analysis and results for an improved multirotor helicopter state estimator. The filter generates more accurate estimates by using an improved dynamic model for the vehicle and by properly accounting for the correlations that exist in the uncertainty during state propagation. As a result, the filter can rely more heavily on frequent and easy to process measurements from gyroscopes and accelerometers, making it more robust to error in the processing intensive information received from the exteroceptive sensors. In Chapter 3 we present BERT, a novel approach to map optimization. The goal of map optimization is to produce an accurate global map of the environment by refining the relative pose transformation estimates generated by the real-time navigation system. We develop BERT to jointly optimize the global poses and relative transformations. BERT exploits properties of independence and conditional independence to allow new information to efficiently flow through the network of transformations. We show that BERT achieves the same final solution as a leading iterative optimization algorithm. However, BERT delivers noticeably better intermediate results for the relative transformation estimates. The improved intermediate results, along with more readily available covariance estimates, make BERT especially applicable to our problem where computational resources are limited. We conclude in Chapter 4 with analysis and results that extend BERT beyond the simple example of Chapter 3. We identify important structure in the network of transformations and address challenges arising in more general map optimization problems. We demonstrate results from several variations of the algorithm and conclude the dissertation with a roadmap for future work.

This thesis is divided into two parts dealing with the optimization problems in Markov decision processes (MDPs) and different neural network-based numerical solvers for partial differential equations (PDEs). In Part I we analyze the optimization problem arising in (partially observable) Markov decision processes using tools from algebraic statistics and information geometry, which can be viewed as neighboring fields of applied algebra and differential geometry, respectively. Here, we focus on infinite horizon problems and memoryless stochastic policies. Markov decision processes provide a mathematical framework for sequential decision-making on which most current reinforcement learning algorithms are built. They formalize the task of optimally controlling the state of a system through appropriate actions. For fully observable problems, the action can be selected knowing the current state of the system. This case has been studied extensively and optimizing the action selection is known to be equivalent to solving a linear program over the (generalized) stationary distributions of the Markov decision process, which are also referred to as state-action frequencies. In Chapter 3, we study partially observable problems where an action must be chosen based solely on an observation of the current state, which might not fully reveal the underlying state. We characterize the feasible state-action frequencies of partially observable Markov decision processes by polynomial inequalities. In particular, the optimization problem in partially observable MDPs is described as a polynomially constrained linear objective program that generalizes the (dual) linear programming formulation of fully observable problems. We use this to study the combinatorial and algebraic complexity of this optimization problem and to upper bound the number of critical points over the individual boundary components of the feasible set. Furthermore, we show that our polynomial programming formulation can be used to effectively solve partially observable MDPs using interior point methods, numerical algebraic techniques, and convex relaxations. Gradient-based methods, including variants of natural gradient methods, have gained tremendous attention in the theoretical reinforcement learning community, where they are commonly referred to as (natural) policy gradient methods. In Chapter 4, we provide a unified treatment of a variety of natural policy gradient methods for fully observable problems by studying their state-action frequencies from the standpoint of information geometry. For a variety of NPGs and reward functions, we show that the trajectories in state-action space are solutions of gradient flows with respect to Hessian geometries, based on which we obtain global convergence guarantees and convergence rates. In particular, we show linear convergence for unregularized and regularized NPG flows with the metrics proposed by Morimura and co-authors and Kakade by observing that these arise from the Hessian geometries of the entropy and conditional entropy, respectively. Further, we obtain sublinear convergence rates for Hessian geometries arising from other convex functions like log-barriers. We provide experimental evidence indicating that our predicted rates are essentially tight. Finally, we interpret the discrete-time NPG methods with regularized rewards as inexact Newton methods if the NPG is defined with respect to the Hessian geometry of the regularizer. This yields local quadratic convergence rates of these methods for step size equal to the inverse penalization strength, which recovers existing results as special cases. Part II addresses neural network-based PDE solvers that have recently experienced tremendous growth in popularity and attention in the scientific machine learning community. We focus on two approaches that represent the approximation of a solution of a PDE as the minimization over the parameters of a neural network: the deep Ritz method and physically informed neural networks. In Chapter 5, we study the theoretical properties of the boundary penalty for these methods and obtain a uniform convergence result for the deep Ritz method for a large class of potentially nonlinear problems. For linear PDEs, we estimate the error of the deep Ritz method in terms of the optimization error, the approximation capabilities of the neural network, and the strength of the penalty. This reveals a trade-off in the choice of the penalization strength, where too little penalization allows large boundary values, and too strong penalization leads to a poor solution of the PDE inside the domain. For physics-informed networks, we show that when working with neural networks that have zero boundary values also the second derivatives of the solution are approximated whereas otherwise only lower-order derivatives are approximated. In Chapter 6, we propose energy natural gradient descent, a natural gradient method with respect to second-order information in the function space, as an optimization algorithm for physics-informed neural networks and the deep Ritz method. We show that this method, which can be interpreted as a generalized Gauss-Newton method, mimics Newton’s method in function space except for an orthogonal projection onto the tangent space of the model. We show that for a variety of PDEs, natural energy gradients converge rapidly and approximations to the solution of the PDE are several orders of magnitude more accurate than gradient descent, Adam and Newton’s methods, even when these methods are given more computational time.:Chapter 1. Introduction 1 1.1 Notation and conventions 7 Part I. Geometry of Markov decision processes 11 Chapter 2. Background on Markov decision processes 12 2.1 State-action frequencies 19 2.2 The advantage function and Bellman optimality 23 2.3 Rational structure of the reward and an explicit line theorem 26 2.4 Solution methods for Markov decision processes 35 Chapter 3. State-action geometry of partially observable MDPs 44 3.1 The state-action polytope of fully observables systems 45 3.2 State-action geometry of partially observable systems 54 3.3 Number and location of critical points 69 3.4 Reward optimization in state-action space (ROSA) 83 Chapter 4. Geometry and convergence of natural policy gradient methods 94 4.1 Natural gradients 96 4.2 Natural policy gradient methods 101 4.3 Convergence of natural policy gradient flows 107 4.4 Locally quadratic convergence for regularized problems 128 4.5 Discussion and outlook 131 Part II. Neural network-based PDE solvers 133 Chapter 5. Theoretical analysis of the boundary penalty method for neural network-based PDE solvers 134 5.1 Presentation and discussion of the main results 137 5.2 Preliminaries regarding Sobolev spaces and neural networks 146 5.3 Proofs regarding uniform convergence for the deep Ritz method 150 5.4 Proofs of error estimates for the deep Ritz method 156 5.5 Proofs of implications of exact boundary values in residual minimization 167 Chapter 6. Energy natural gradients for neural network-based PDE solvers 174 6.1 Energy natural gradients 176 6.2 Experiments 183 6.3 Conclusion and outlook 192 Bibliography 193

This dissertation addresses attitude control and inertial navigation of autonomous underwater vehicles (AUVs). We present theoretical justification for using simplified models, derive system identification algorithms, and verify our results through extensive field trials. Although this research focuses on small AUVs with limited instrumentation, the results are useful for underwater vehicles of any size. For attitude control of aircraft systems, second-order equivalent pitch-axis models are common and extensively studied. However, similar analysis has not been performed for the pitch-axis motion of underwater vehicles. In this dissertation, we study the utility and the limitations of second-order approximate models for AUVs. We seek to improve the flight performance and shorten the time required to re-design a control algorithm when the shape, mass-distribution, and/or net buoyancy of an AUV/payload configuration changes. In comparison to commonly implemented AUV attitude controllers, which neglect roll motion and address pitch and yaw dynamics separately, we derive a novel linear time-varying model that explicitly displays the coupling between pitch and yaw motion due to nonzero roll angle and/or roll rate. The model facilitates an Hâ control design approach that explicitly addresses robustness against those coupling terms and significantly reduces the effect of pitch and yaw coupling. To improve AUV navigation, we investigate algorithms for calibrating a triaxial gyroscope using angular orientation measurements and formally define AUV trajectories that are persistently exciting and for which the calibration coefficients are uniformly observable. To improve AUV guidance, we propose a near real-time ocean current identification method that estimates a non-uniform flow-field using only sparse flow measurements. / Ph. D.

Monitoring and protection of power systems is a task that has manifold objectives. Amongst others, it involves performing data mining, optimizing available resources, assessing system stresses, and doing data conditioning. The role of PMUs in fulfilling these four objectives forms the basis of this dissertation. Classification and regression tree (CART) built using phasor data has been extensively used in power systems. The splits in CART are based on a single attribute or a combination of variables chosen by CART itself rather than the user. But as PMU data consists of complex numbers, both the attributes, should be considered simultaneously for making critical decisions. An algorithm is proposed here that expresses high dimensional, multivariate data as a single attribute in order to successfully perform splits in CART. In order to reap maximum benefits from placement of PMUs in the power grid, their locations must be selected judiciously. A gradual PMU placement scheme is developed here that ensures observability as well as protects critical parts of the system. In order to circumvent the computational burden of the optimization, this scheme is combined with a topology-based system partitioning technique to make it applicable to virtually any sized system. A power system is a dynamic being, and its health needs to be monitored at all times. Two metrics are proposed here to monitor stress of a power system in real-time. Angle difference between buses located across the network and voltage sensitivity of buses lying in the middle are found to accurately reflect the static and dynamic stress of the system. The results indicate that by setting appropriate alerts/alarm limits based on these two metrics, a more secure power system operation can be realized. A PMU-only linear state estimator is intrinsically superior to its predecessors with respect to performance and reliability. However, ensuring quality of the data stream that leaves this estimator is crucial. A methodology for performing synchrophasor data conditioning and validation that fits neatly into the existing linear state estimation formulation is developed here. The results indicate that the proposed methodology provides a computationally simple, elegant solution to the synchrophasor data quality problem. / Ph. D.

Linux som operativsystem används mer och har blivit vanligare i samband med inbyggda system vilket har lett till att företag som Percepio ställer frågor om hur datakollektion vid processkrascher genomförs i Linux-baserade inbyggda system. Detta med anledning för att öka observerbarhet i system via externa medel och tillåta fjärrfelsökning via en molnbaserad informationspanel. Syftet med det här arbetet är att undersöka vad det finns för befintliga tillvägagångssätt kring datainsamling vid signalavbrott. Vårt arbete har som mål att implementera en prototyp av Percepios övervakningsverktyg DevAlert på ett Linux-baserat inbyggt system. I den här rapporten kommer vi att undersöka hur praxisen ser ut för felsökning och felhantering i den här typen av system. För att uppfylla syfte och mål med vårt arbete har vi samlat information i en litteraturstudie om vad som är relevant för att öka observerbarheten i liknande system som har felande processer. Detta följdes av iterativa experiment där den insamlade informationen från litteraturstudien bekräftats och implementerats i vår prototyp. Den slutliga iterationen utfördes på en virtuell maskin vilket resulterade i en lyckad prototypimplementation av DevAlert i Linux. Resultaten som vi presenterar anser vi ska kunna appliceras i ett inbyggt Linux-system eftersom ramverken som vi samlar information ifrån använder en Linux-kärna. / The Linux operating systems is used more frequently and have become more common in connection with embedded systems, which has led to companies such as Percepio to pose questions about how data collection in case of process crashes is carried out in Linux based embedded systems. This is to increase observability in systems via external means and allow remote troubleshooting via a cloud-based dashboard. The purpose of this work is to investigate what the existing approaches regarding data collection in case of signal interruption are. Our work aims to implement a prototype of Percepio's monitoring tool DevAlert on a Linux-based embedded system. In this report, we will examine what the best practices are for troubleshooting and error handling in this type of system. In order to fulfill the purpose and goals of our work, we have gathered information in a literature study about what is relevant to increase observability in similar systems that have faulty processes. This was followed by iterative experiments where the collected information from the literature study was confirmed and implemented in our prototype. The final iteration was performed on a virtual machine resulting in a successful prototype implementation of DevAlert in Linux. We believe that the results we present should be applicable in an embedded Linux system because the frameworks from which we gather information use a Linux kernel.

This dissertation investigates high-level decision making for agents that are both goal and utility driven. We develop a partially observable Markov decision process (POMDP) planner which is an extension of an agent programming language called DTGolog, itself an extension of the Golog language. Golog is based on a logic for reasoning about action—the situation calculus. A POMDP planner on its own cannot cope well with dynamically changing environments and complicated goals. This is exactly a strength of the belief-desire-intention (BDI) model: BDI theory has been developed to design agents that can select goals intelligently, dynamically abandon and adopt new goals, and yet commit to intentions for achieving goals. The contribution of this research is twofold: (1) developing a relational POMDP planner for cognitive robotics, (2) specifying a preliminary BDI architecture that can deal with stochasticity in action and perception, by employing the planner. / Computing / M. Sc. (Computer Science)

Les systèmes multi-capteurs exploitent les complémentarités des différentes sources sensorielles. Par exemple, le capteur visuo-inertiel permet d’estimer la pose à haute fréquence et avec une grande précision. Les méthodes de vision mesurent la pose à basse fréquence mais limitent la dérive causée par l’intégration des données inertielles. Les centrales inertielles mesurent des incréments du déplacement à haute fréquence, ce que permet d’initialiser la vision et de compenser la perte momentanée de celle-ci. Cette thèse analyse deux aspects du problème. Premièrement, nous étudions les méthodes visuelles directes pour l’estimation de pose, et proposons une nouvelle technique basée sur la corrélation entre des images et la pondération des régions et des pixels, avec une optimisation inspirée de la méthode de Newton. Notre technique estime la pose même en présence des changements d’illumination extrêmes. Deuxièmement, nous étudions la fusion des données a partir de la théorie de la commande. Nos résultats principaux concernent le développement d’observateurs pour l’estimation de pose, biais IMU et l’autocalibrage. Nous analysons la dynamique de rotation d’un point de vue non linéaire, et fournissons des observateurs stables dans le groupe des matrices de rotation. Par ailleurs, nous analysons la dynamique de translation en tant que système linéaire variant dans le temps, et proposons des conditions d’observabilité uniforme. Les analyses d’observabilité nous permettent de démontrer la stabilité uniforme des observateurs proposés. La méthode visuelle et les observateurs sont testés et comparés aux méthodes classiques avec des simulations et de vraies données visuo-inertielles. / Systems with multiple sensors can provide information unavailable from a single source, and complementary sensory characteristics can improve accuracy and robustness to many vulnerabilities as well. Explicit pose measurements are often performed either with high frequency or precision, however visuo-inertial sensors present both features. Vision algorithms accurately measure pose at low frequencies, but limit the drift due to integration of inertial data. Inertial measurement units yield incremental displacements at high frequencies that initialize vision algorithms and compensate for momentary loss of sight. This thesis analyzes two aspects of that problem. First, we survey direct visual tracking methods for pose estimation, and propose a new technique based on the normalized crosscorrelation, region and pixel-wise weighting together with a Newton-like optimization. This method can accurately estimate pose under severe illumination changes. Secondly, we investigate the data fusion problem from a control point of view. Main results consist in novel observers for concurrent estimation of pose, IMU bias and self-calibration. We analyze the rotational dynamics using tools from nonlinear control, and provide stable observers on the group of rotation matrices. Additionally, we analyze the translational dynamics using tools from linear time-varying systems, and propose sufficient conditions for uniform observability. The observability analyses allow us to prove uniform stability of the observers proposed. The proposed visual method and nonlinear observers are tested and compared to classical methods using several simulations and experiments with real visuo-inertial data.

Neste trabalho, introduzimos os conceitos de controlabilidade e de observabilidade para equações diferenciais ordinárias generalizadas, apresentamos resultados inéditos sobre condições suficientes e necessárias para controlabilidade e para observabilidade para estas equações e também apresentaremos uma aplicação. Utilizando teoremas de correspondência entre equações diferenciais ordinárias generalizadas e outras equações diferenciais, traduzimos os resultados obtidos para os casos particulares de controlabilidade e observabilidade para equações diferenciais em medida e equações diferencias com impulsos. O fato de trabalharmos no ambiente das equações diferenciais ordinárias generalizadas permitiu que os resultados obtidos pudessem envolver funções com muitas descontinuidades e muito oscilantes, ou seja, de variação ilimitada. Os resultados novos apresentados aqui estão contidos no artigo [21] que se encontra em fase final de redação e será submetido à publicação em breve. / In this work, we introduce concepts of controllability and observability for generalized ordinary differential equations, we present new results on necessary and sufficient conditions for controllability and observability for these equations and we also present an application. Using theorems of correspondence between generalized ordinary differential equations and other differential equations, we translate the results obtained for the particular cases of controllability and observability for measure differential equations and differential equations with impulses. The fact that we work in the framework of generalized ordinary differential equations allows us to obtain results where the functions involved can have many discontinuities and be highly oscillating, that is, of unbounded variation. The new results presented here are contained in the preprint [21] which is under final revision and will soon be submitted for publication.