The main objective of this thesis is to apply ideas and techniques from modern control theory, especially from robust state estimation and model validation, to various important problems in computer vision. Robust model validation is used in texture recognition where new approaches for classifying texture samples and segmenting textured images are developed. Also, a new model validation approach to motion primitive recognition is demonstrated by considering the motion segmentation problem for a mobile wheeled robot. A new approach to image inpainting based on robust state estimation is proposed where the implementation presented here concerns with recovering corrupted frames in video sequences. Another application addressed in this thesis based on robust state estimation is video-based tracking. A new tracking system is proposed to follow connected regions in video frames representing the objects in consideration. The system accommodates tracking multiple objects and is designed to be robust towards occlusions. To demonstrate the performance of the proposed solutions, examples are provided where the developed methods are applied to various gray-scale images, colored images, gray-scale videos and colored videos. In addition, a new algorithm is introduced for motion estimation via inverse polynomial interpolation. Motion estimation plays a primary role within the video-based tracking system proposed in this thesis. The proposed motion estimation algorithm is also applied to medical image sequences. Motion estimation results presented in this thesis include pairs of images from a echocardiography video and a robot-assisted surgery video.

The main objective of this thesis is to apply ideas and techniques from modern control theory, especially from robust state estimation and model validation, to various important problems in computer vision. Robust model validation is used in texture recognition where new approaches for classifying texture samples and segmenting textured images are developed. Also, a new model validation approach to motion primitive recognition is demonstrated by considering the motion segmentation problem for a mobile wheeled robot. A new approach to image inpainting based on robust state estimation is proposed where the implementation presented here concerns with recovering corrupted frames in video sequences. Another application addressed in this thesis based on robust state estimation is video-based tracking. A new tracking system is proposed to follow connected regions in video frames representing the objects in consideration. The system accommodates tracking multiple objects and is designed to be robust towards occlusions. To demonstrate the performance of the proposed solutions, examples are provided where the developed methods are applied to various gray-scale images, colored images, gray-scale videos and colored videos. In addition, a new algorithm is introduced for motion estimation via inverse polynomial interpolation. Motion estimation plays a primary role within the video-based tracking system proposed in this thesis. The proposed motion estimation algorithm is also applied to medical image sequences. Motion estimation results presented in this thesis include pairs of images from a echocardiography video and a robot-assisted surgery video.

Kalman-type filtering methods are mostly designed based on exact knowledge of the system’s model with known parameters. In real applications, there may be considerable amount of uncertainties about the model structure, physical parameters, level of noise, and initial conditions. In order to overcome such difficulties, robust state estimation techniques are recommended. This PhD thesis presents a novel robust state estimation method that is referred to as the 2nd-order smooth variable structure filter (2nd-order SVSF) and satisfies the first and second order sliding conditions. It is an extension to the 1st-order SVSF introduced in 2007. In the 1st-order SVSF chattering is reduced by using a smoothing boundary layer; however, the 2nd-order SVSF alleviates chattering by preserving the second order sliding condition. It reduces the estimation error and its first difference until the existence boundary layer is reached. Then after, it guarantees that the estimation error and its difference remain bounded given bounded noise and modeling uncertainties. As such, the 2nd-order SVSF produces more accurate and smoother state estimates under highly uncertain conditions than the 1st-order version. The main issue with the 2nd-order SVSF is that it is not optimal in the mean square error sense. In order to overcome this issue, the dynamic 2nd-order SVSF is initially presented based on a dynamic sliding mode manifold. This manifold introduces a variable cut-off frequency coefficient that adjusts the filter bandwidth. An optimal derivation of the 2nd-order SVSF is then obtained by minimizing the state error covariance matrix with respect to the cut-off frequency matrix. An experimental setup of an electro-hydrostatic actuator is used to compare the performance of the 2nd-order SVSF and its optimal version with other estimation methods such as the Kalman filter and the 1st-order SVSF. Experiments confirm the superior performance of the 2nd-order SVSF given modeling uncertainties. / Thesis / Doctor of Philosophy (PhD)

Robust estimation methods have been successfully applied to the problem of power system state estimation in a real-time environment. The Schweppe-type GM-estimator with the Huber psi-function (SHGM) has been fully installed in conjunction with a topology processor in the EEF utility, headquartered in Fribourg, Switzerland. Some basic concepts of maximum likelihood estimation and robust analysis are reviewed, and applied to the development of the SHGM-estimator. The algorithms used by the topology processor and state estimator are presented, and the superior performance of the SHGM-estimator over the classic weighted least squares estimator is demonstrated on the EEF network. The measurement configuration of the EEF network has been evaluated, and suggestions for its reinforcement have been proposed. / Master of Science

A naive approach to filtering for feedback control of dynamic systems that is robust and stable is proposed. Simulations are run on the filters presented to investigate the robustness properties of each filter. Each simulation with the comparison of the filters is carried out using the usual mean squared error. The filters to be included are the classic Kalman filter, Krein space Kalman, two adjustments to the Krein filter with input modeling and a second uncertainty parameter, a newly developed filter called the Naive filter, bias corrected Naive, exponentially weighted moving average (EWMA) Naive, and bias corrected EWMA Naive filter.

Indirect wind estimation onboard unmanned aerial systems (UASs) can be accomplished using existing air vehicle sensors along with a dynamic model of the UAS augmented with additional wind-related states. It is often desired to extract a mean component of the wind the from frequency fluctuations (i.e. turbulence). Commonly, a variation of the KALMAN filter is used, with explicit or implicit assumptions about the nature of the random wind velocity. This dissertation presents an H-infinity (H∞) filtering approach to wind estimation which requires no assumptions about the statistics of the process or measurement noise. To specify the wind frequency content of interest a low-pass filter is incorporated. We develop the augmented UAS model in continuous-time, derive the H∞ filter, and introduce a KALMAN-BUCY filter for comparison. The filters are applied to data gathered during UAS flight tests and validated using a vaned air data unit onboard the aircraft. The H∞ filter provides quantitatively better estimates of the wind than the KALMAN-BUCY filter, with approximately 10-40% less root-mean-square (RMS) error in the majority of cases. It is also shown that incorporating DRYDEN turbulence does not improve the KALMAN-BUCY results. Additionally, this dissertation describes the theory and process for using generalized polynomial chaos (gPC) to re-cast the dynamics of a system with non-deterministic parameters as a deterministic system. The concepts are applied to the problem of wind estimation and characterizing the precision of wind estimates over time due to known parametric uncertainties. A novel truncation method, known as Sensitivity-Informed Variable Reduction (SIVR) was developed. In the multivariate case presented here, gPC and the SIVR-derived reduced gPC (gPCr) exhibit a computational advantage over Monte Carlo sampling-based methods for uncertainty quantification (UQ) and sensitivity analysis (SA), with time reductions of 38% and 98%, respectively. Lastly, while many estimation approaches achieve desirable accuracy under the assumption of known system parameters, reducing the effect of parametric uncertainty on wind estimate precision is desirable and has not been thoroughly investigated. This dissertation describes the theory and process for combining gPC and H-infinity (H∞) filtering. In the multivariate case presented, the gPC H∞ filter shows superiority over a nominal H∞ filter in terms of variance in estimates due to model parametric uncertainty. The error due to parametric uncertainty, as characterized by the variance in estimates from the mean, is reduced by as much as 63%. / Doctor of Philosophy / On unmanned aerial systems (UASs), determining wind conditions indirectly, without direct measurements, is possible by utilizing onboard sensors and computational models. Often, the goal is to isolate the average wind speed while ignoring turbulent fluctuations. Conventionally, this is achieved using a mathematical tool called the KALMAN filter, which relies on assumptions about the wind. This dissertation introduces a novel approach called H-infinity (H∞) filtering, which does not rely on such assumptions and includes an additional mechanism to focus on specific wind frequencies of interest. The effectiveness of this method is evaluated using real-world data from UAS flights, comparing it with the traditional KALMAN-BUCY filter. Results show that the H∞ filter provides significantly improved wind estimates, with approximately 10-40% less error in most cases. Furthermore, the dissertation addresses the challenge of dealing with uncertainty in wind estimation. It introduces another mathematical technique called generalized polynomial chaos (gPC), which is used to quantify and manage uncertainties within the UAS system and their impact on the indirect wind estimates. By applying gPC, the dissertation shows that the amount and sources of uncertainty can be determined more efficiently than by traditional methods (up to 98% faster). Lastly, this dissertation shows the use of gPC to provide more precise wind estimates. In experimental scenarios, employing gPC in conjunction with H∞ filtering demonstrates superior performance compared to using a standard H∞ filter alone, reducing errors caused by uncertainty by as much as 63%.

<p>The use of lower airspace for air taxi and cargo applications opens up exciting prospects for futuristic Unmanned Aircraft Systems (UAS). However, ensuring the safety and security of these UAS within densely populated urban areas presents significant challenges. Most modern aircraft systems, whether unmanned or otherwise, rely on the Global Navigation Satellite System (GNSS) as a primary sensor for navigation. From satellite navigations point of view, the dense urban environment compromises positioning accuracy due to signal interference, multipath effects, etc. Furthermore, civilian GNSS receivers are susceptible to spoofing attacks since they lack encryption capabilities. Therefore, in this thesis, we focus on examining the safety and cybersecurity assurance of UAS in dense urban environments, from both theoretical and experimental perspectives. </p> <p>To facilitate the verification and validation of the UAS, the first part of the thesis focuses on the development of a realistic GNSS sensor emulation using a Gazebo plugin. This plugin is designed to replicate the complex behavior of the GNSS sensor in urban settings, such as multipath reflections, signal blockages, etc. By leveraging the 3D models of the urban environments and the ray-tracing algorithm, the plugin predicts the spatial and temporal patterns of GNSS signals in densely populated urban environments. The efficacy of the plugin is demonstrated for various scenarios including routing, path planning, and UAS cybersecurity. </p> <p>Subsequently, a robust state estimation algorithm for dynamical systems whose states can be represented by Lie Groups (e.g., rigid body motion) is presented. Lie groups provide powerful tools to analyze the complex behavior of non-linear dynamical systems by leveraging their geometrical properties. The algorithm is designed for time-varying uncertainties in both the state dynamics and the measurements using the log-linear property of the Lie groups. When unknown disturbances are present (such as GNSS spoofing, and multipath effects), the log-linearization of the non-linear estimation error dynamics results in a non-linear evolution of the linear error dynamics. The sufficient conditions under which this non-linear evolution of estimation error is bounded are derived, and Lyapunov stability theory is employed to design a robust filter in the presence of an unknown-but-bounded disturbance. </p>

Nombreux sont les problèmes classiquement rencontrés dans les sciences de l'ingénieur dont la résolution fait appel à l'estimation d'état d'un système par le biais d'un observateur. La synthèse d'un observateur n'est envisageable qu'à la condition de disposer d'un modèle à la fois exploitable et représentatif du comportement dynamique du système. Or, la modélisation du système et la synthèse de l'observateur deviennent des tâches difficiles à accomplir dès lors que le comportement dynamique du système doit être représenté par un modèle de nature non linéaire. Face à ces difficultés, l'approche multimodèle peut être mise à profit. Les travaux présentés dans cette thèse portent sur les problèmes soulevés par l'identification, l'estimation d'état et le diagnostic de systèmes non linéaires représentés à l'aide d'un multimodèle découplé. Ce dernier, composé de sous-modèles qui peuvent être de dimensions différentes, est doté d'un haut degré de généralité et de flexibilité et s'adapte particulièrement bien à la modélisation des systèmes complexes à structure variable. Cette caractéristique le démarque des approches multimodèles plus conventionnelles qui ont recours à des sous-modèles de même dimension. Après une brève introduction à l'approche multimodèle, le problème de l'estimation paramétrique du multimodèle découplé est abordé. Puis sont présentés des algorithmes de synthèse d'observateurs d'état robustes vis-à-vis des perturbations, des incertitudes paramétriques et des entrées inconnues affectant le système. Ces algorithmes sont élaborés à partir de trois types d'observateurs dits à gain proportionnel, à gain proportionnel-intégral et à gain multi-intégral. Enfin, les différentes phases d'identification, de synthèse d'observateurs et de génération d'indicateurs de défauts sont illustrées au moyen d'un exemple académique de diagnostic du fonctionnement d'un bioréacteur / The state estimation of a system, with the help of an observer, is largely used in many practical situations in order to cope with many classic problems arising in control engineering. The observer design needs an exploitable model able to give an accurate description of the dynamic behaviour of the system. However, system modelling and observer design can not easily be accomplished when the dynamic behaviour of the system must be described by non linear models. The multiple model approach can be used to tackle these difficulties. This thesis deals with black box modelling, state estimation and fault diagnosis of nonlinear systems represented by a decoupled multiple model. This kind of multiple model provides a high degree of generality and flexibility in the modelling stage. Indeed, the decoupled multiple model is composed of submodels which dimensions can be different. Thus, this feature is a significant difference between the decoupled multiple model and the classical used multiple model where all the submodels have the same dimension. After a brief introduction to the multiple model approach, the parametric identification problem of a decoupled multiple model is explored. Algorithms for robust observers synthesis with respect to perturbations, modelling uncertainties and unknown inputs are afterwards presented. These algorithms are based on three kinds of observers called proportional, proportional-integral and multiple-integral. Lastly, identification, observers synthesis and fault sensitivity signals generation are illustrated via a simulation example of a bioreactor

Cette thèse traite le problème de l'estimation d'état, du diagnostic et de commande tolérante aux défauts des systèmes non linéaires représentés par un modèle de Takagi-Sugeno (T-S) à variables de prémisse non mesurables. De nombreux algorithmes pour la synthèse d'observateurs robustes vis-à-vis des perturbations, des imperfections de modélisation et des entrées inconnues sont présentés en se basant sur quatre types d'observateurs : les observateursproportionnels, les observateurs à entrées inconnues, les observateurs proportionnel intégral (PI) et multi-intégral (PMI). Par la suite, ces derniers sont utilisés pour le diagnostic de fautes affectant des systèmes non linéaires. Ceci est réalisé au moyen de trois stratégies. La première utilise l'observateur à entrée inconnue par découplage afin de rendre l'observateur insensible à certains défauts et permettre de détecter et d'isoler les défauts en construisant des bancs d'observateurs. En raison des conditions structurelles souvent insatisfaites, le découplage total des défauts de l'erreur d'estimation d'état n'est pas réalisable. Afin de s'affranchir de ces contraintes, la seconde stratégie utilise les observateurs PI et PMI pour estimer simultanément l'état et les défauts du système. La troisième stratégie qui utilise le formalisme H8 vise à concevoir un générateur de résidus minimisant l'influence des perturbations et maximisant l'influence des défauts. Un choix adéquat des paramètres du générateur de résidus permet la détection, la localisation et l'estimation des défauts. Enfin, une loi de commande tolérante aux défauts par poursuite de trajectoire d'un modèle de référence estproposée en exploitant les observateurs PI et PMI / This thesis deals with state estimation, fault diagnosis and fault tolerant control of nonlinear systems represented by a Takagi-Sugeno model with unmeasurable premise variables. The problem of state estimation of nonlinear systems with T-S model with unmeasurable premise variable is explored. Algorithms for robust observers synthesis with respect to perturbations, modeling uncertainties and unknown inputs are afterward presented. These algorithms are based on four kinds of observers called proportional, unknown input observers (UIOs), proportional-integral (PI) and multiple-integral (PMI) . The application on model-based diagnosis is studied based on three strategies. The first one uses unknown input observer to decouple some faults and makes the observers insensitive to certain faults. This allows to detect and isolate faults by constructing observers banks. Due to strong structural conditions on designing UIOs decoupling the faults on the state estimation error is not possible. To avoid this problem, the second strategy uses PI and PMI observers in order to estimate simultaneously the state and the faults of the system. The third strategy uses the H8 formalism. This aims to minimize the influence of perturbations and to maximize the effects of faults on the residual signal. An adequate choice of the residual generator parameters allows to detect, to isolate and to estimate the faults affecting the system. Lastly, a fault tolerant control law is proposed by reference trajectory tracking based on the use of PI and PMI observers

Robust state estimation solutions must deal with faulty measurements, called outliers, and unknown data associations, which lead to multiple feasible hypotheses. Take, for instance, the scenario of tracking two indistinguishable targets based on position measurements, where each measurement could refer to either of the targets or even be a faulty reading. Common estimation methods model the state as having a unimodal distribution, so they are called unimodal methods. Likewise, multimodal methods model the state as a multimodal distribution. Difficult problems, such as autonomous underwater vehicle (AUV) navigation relying on acoustic localization, frequently involve recurring outliers. In these situations, the correct hypothesis only emerges as the most likely one when a substantial number of measurements are considered. Robust solutions for these problems need to consider multiple hypotheses simultaneously, which, in turn, calls for the representation of multimodal distributions. In this work, a novel approximate inference method is presented, called the Gaussian mixture sum-product algorithm (GM-SPA), as it implements the sum-product algorithm (SPA) for Gaussian mixtures. The GM-SPA can exactly represent under-constrained linear measurements and approximate important non-linear models, such as range measurements and 2D pose kinematics. The outlier robustness of the GM-SPA is tested and compared against the particle filter (PF) and multimodal incremental smoothing and mapping (MMiSAM), both of which are non-parametric methods. Robustness, accuracy, and run-time are improved in simulation tests. The test problems include 1D localization with unknown data association, 3D linear target tracking with correlated outliers, and 2D range-only pose estimation with Gaussian mixture noise. / Robusta lösningar för tillståndsuppskattning måste kunna hantera felaktiga mätningar, så kallade outliers, och okända dataassociationer, vilket leder till flera möjliga hypoteser. Ta till exempel scenariot att spåra två likadana mål baserat på positionsmätningar, där varje mätning kan tillhöra något av målen eller till och med vara en felaktig avläsning. Vanliga skattningsmetoder modellerar tillståndet som en unimodal fördelning, och kallas därför unimodala metoder. På samma sätt modellerar multimodala metoder tillståndet som en multimodal fördelning. Svåra problem, som navigering av autonoma undervattensfarkoster (AUV) med hjälp av akustisk lokalisering, involverar ofta upprepade outliers. I dessa situationer framstår den korrekta hypotesen som den mest sannolika först när ett stort antal mätningar beaktas. Robusta lösningar för dessa problem måste ta hänsyn till flera hypoteser samtidigt, vilket i sin tur kräver representation av multimodala fördelningar. I detta arbete presenteras en ny approximativ inferensmetod, kallad Gaussian mixture sum-product algorithm (GM-SPA), eftersom den implementerar sum-product algorithm (SPA) för gaussiska blandningar. GM-SPA kan representera underbegränsade linjära mätningar exakt och approximera viktiga icke-linjära modeller, till exempel avståndsmätningar eller 2D-posekinematik. GM-SPA:s robusthet mot outliers testas och jämförs med partikelfiltret (PF) och multimodal incremental smoothing and mapping (MM-iSAM), som båda är icke-parametriska metoder. Robusthet, noggrannhet och körtid förbättras i simuleringstester. Simulerade tester inkluderar 1D-lokalisering med okänd dataassociation, 3D linjär målföljning med korrelerade outliers och 2D-ställningsuppskattning av endast räckvidd med Gaussiskt blandningsljud. / Soluções robustas para estimação de estado devem lidar com medidas defeituosas, chamadas de outliers, e com associações de dados desconhecidas, que levam a múltiplas hipóteses possíveis. Considere-se, por exemplo, o cenário de rastreamento de dois alvos indistinguíveis com base em medidas de posição, em que cada medida pode-se referir a qualquer um dos alvos ou até mesmo ser uma leitura defeituosa. Métodos de estimação comuns modelam o estado como tendo uma distribuição unimodal, sendo assim chamados de métodos unimodais. Da mesma forma, métodos multimodais modelam o estado como uma distribuição multimodal. Problemas difíceis, como a navegação de veículos subaquáticos autónomos (AUVs) baseada em localização acústica, frequentemente envolvem outliers recorrentes. Nestas situações, a hipótese correta apenas surge como a mais provável quando um número substancial de medidas é considerado. Soluções robustas para estes problemas precisam de considerar múltiplas hipóteses simultaneamente, o que, por sua vez, exige a representação de distribuições multimodais. Neste trabalho, é apresentado um novo método de inferência aproximada, chamado Gaussian mixture sum-product algorithm (GM-SPA), pois implementa o sum-product algorithm (SPA) para misturas Gaussianas. O GM-SPA pode representar exatamente medidas lineares sub-determinadas e aproximar modelos não lineares importantes, como medidas de distância e cinemática de pose 2D. A robustez a outliers do GM-SPA é testada e comparada com o filtro de partículas (PF) e com multimodal incremental smoothing and mapping (MM- -iSAM), ambos métodos não-paramétricos. A robustez, a exatidão e o tempo de execução em testes de simulação são melhorados. Os problemas de teste incluem localização 1D com associação de dados desconhecida, rastreamento linear de alvos em 3D com outliers correlacionados e estimação de pose 2D com base em medidas de distância com ruído de mistura Gaussiana.