Making decisions and predictions from noisy observations are two important and challenging problems in many areas of society. Some examples of applications are recommendation systems for online shopping and streaming services, connecting genes with certain diseases and modelling climate change. In this thesis, we make use of Bayesian statistics to construct probabilistic models given prior information and historical data, which can be used for decision support and predictions. The main obstacle with this approach is that it often results in mathematical problems lacking analytical solutions. To cope with this, we make use of statistical simulation algorithms known as Monte Carlo methods to approximate the intractable solution. These methods enjoy well-understood statistical properties but are often computational prohibitive to employ. The main contribution of this thesis is the exploration of different strategies for accelerating inference methods based on sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC). That is, strategies for reducing the computational effort while keeping or improving the accuracy. A major part of the thesis is devoted to proposing such strategies for the MCMC method known as the particle Metropolis-Hastings (PMH) algorithm. We investigate two strategies: (i) introducing estimates of the gradient and Hessian of the target to better tailor the algorithm to the problem and (ii) introducing a positive correlation between the point-wise estimates of the target. Furthermore, we propose an algorithm based on the combination of SMC and Gaussian process optimisation, which can provide reasonable estimates of the posterior but with a significant decrease in computational effort compared with PMH. Moreover, we explore the use of sparseness priors for approximate inference in over-parametrised mixed effects models and autoregressive processes. This can potentially be a practical strategy for inference in the big data era. Finally, we propose a general method for increasing the accuracy of the parameter estimates in non-linear state space models by applying a designed input signal. / Borde Riksbanken höja eller sänka reporäntan vid sitt nästa möte för att nå inflationsmålet? Vilka gener är förknippade med en viss sjukdom? Hur kan Netflix och Spotify veta vilka filmer och vilken musik som jag vill lyssna på härnäst? Dessa tre problem är exempel på frågor där statistiska modeller kan vara användbara för att ge hjälp och underlag för beslut. Statistiska modeller kombinerar teoretisk kunskap om exempelvis det svenska ekonomiska systemet med historisk data för att ge prognoser av framtida skeenden. Dessa prognoser kan sedan användas för att utvärdera exempelvis vad som skulle hända med inflationen i Sverige om arbetslösheten sjunker eller hur värdet på mitt pensionssparande förändras när Stockholmsbörsen rasar. Tillämpningar som dessa och många andra gör statistiska modeller viktiga för många delar av samhället. Ett sätt att ta fram statistiska modeller bygger på att kontinuerligt uppdatera en modell allteftersom mer information samlas in. Detta angreppssätt kallas för Bayesiansk statistik och är särskilt användbart när man sedan tidigare har bra insikter i modellen eller tillgång till endast lite historisk data för att bygga modellen. En nackdel med Bayesiansk statistik är att de beräkningar som krävs för att uppdatera modellen med den nya informationen ofta är mycket komplicerade. I sådana situationer kan man istället simulera utfallet från miljontals varianter av modellen och sedan jämföra dessa mot de historiska observationerna som finns till hands. Man kan sedan medelvärdesbilda över de varianter som gav bäst resultat för att på så sätt ta fram en slutlig modell. Det kan därför ibland ta dagar eller veckor för att ta fram en modell. Problemet blir särskilt stort när man använder mer avancerade modeller som skulle kunna ge bättre prognoser men som tar för lång tid för att bygga. I denna avhandling använder vi ett antal olika strategier för att underlätta eller förbättra dessa simuleringar. Vi föreslår exempelvis att ta hänsyn till fler insikter om systemet och därmed minska antalet varianter av modellen som behöver undersökas. Vi kan således redan utesluta vissa modeller eftersom vi har en bra uppfattning om ungefär hur en bra modell ska se ut. Vi kan också förändra simuleringen så att den enklare rör sig mellan olika typer av modeller. På detta sätt utforskas rymden av alla möjliga modeller på ett mer effektivt sätt. Vi föreslår ett antal olika kombinationer och förändringar av befintliga metoder för att snabba upp anpassningen av modellen till observationerna. Vi visar att beräkningstiden i vissa fall kan minska ifrån några dagar till någon timme. Förhoppningsvis kommer detta i framtiden leda till att man i praktiken kan använda mer avancerade modeller som i sin tur resulterar i bättre prognoser och beslut.

Afin de parvenir à l'autonomie des robots mobiles, la localisation efficace est une condition préalable nécessaire. Le suivi de position, la localisation globale et le problème du robot kidnappé sont les trois sous-problèmes que nous étudions. Dans cette thèse, nous comparons en simulation trois algorithmes de localisation Markovienne. Nous proposons ensuite une amélioration de l'algorithme de localisation de Monte Carlo par filtre particulaire. Cet algorithme (nommé SAMCL) utilise des particules auto-adaptatives. En employant une technique de pré-mise en cache pour réduire le temps de calcul en ligne, l'algorithme SAMCL est plus efficace que la méthode de Monte Carlo usuelle. En outre, nous définissons la notion de région d'énergie similaire (SER), qui est un ensemble de poses (cellules de la grille) dont l'énergie-capteur est similaire avec celle du robot dans l'espace réel. En distribuant les échantillons globaux dans SER lieu de les distribuer au hasard dans la carte, SAMCL obtient une meilleure performance dans la localisation et résout ces trois sous-problèmes. La localisation coopérative de plusieurs robots est également étudiée. Nous avons développé un algorithme (nommé PM) pour intégrer l'information de localisation échangée par les robots lors d'une rencontre au cours d'une mission commune. Cet algorithme apparaît comme une extension à l'algorithme de SAMCL et a été validé en simulation. La validité et l'efficacité de notre approche sont démontrées par des expériences sur un robot réel évoluant dans un environnement connu et préalablement cartographié. / In order to achieve the autonomy of mobile robots, effective localization is a necessary prerequisite. In this thesis, we study and compare three regular Markov localization algorithms by simulations. Then we propose an improved Monte Carlo localization algorithm using self-adaptive samples, abbreviated as SAMCL. By employing a pre-caching technique to reduce the on-line computational burden, SAMCL is more efficient than regular MCL. Further, we define the concept of similar energy region (SER), which is a set of poses (grid cells) having similar energy with the robot in the robot space. By distributing global samples in SER instead of distributing randomly in the map, SAMCL obtains a better performance in localization. Position tracking, global localization and the kidnapped robot problem are the three sub-problems of the localization problem. Most localization approaches focus on solving one of these sub-problems. However, SAMCL solves all the three sub-problems together thanks to self-adaptive samples that can automatically separate themselves into a global sample set and a local sample set according to needs. Cooperative localization among multiple robots is carried out by exchanging localization information derived from cooperation. We devise the Position Mapping (PM) algorithm to integrate this information, which can merge into the SAMCL algorithm as an extension. The validity and the efficiency of our algorithms are demonstrated by experiments carried out with a real robot in a structured and known environment.

In this study, accurate target tracking for bearings-only tracking problem is investigated. A new tracking filter for this nonlinear problem is designed where both range parameterization and Rao-Blackwellized (marginalized) particle filtering techniques are used in a Gaussian mixture formulation to track both constant velocity and maneuvering targets. The idea of using target turn rate in the state equation in such a way that marginalization is possible is elaborated. Addition to nonlinear nature, unobservability is a major problem of bearings-only tracking. Observer trajectory generation to increase the observability of the bearings-only tracking problem is studied. Novel formulation of observability measures based on mutual information between the state and the measurement sequences are derived for the problem. These measures are used as objective functions to improve observability. Based on the results obtained better understanding of the required observer trajectory for accurate bearings-only target tracking is developed.

Unmanned aerial vehicles (UAV's) have been the focus of significant research interest in both military and commercial areas since they have a variety of practical applications including reconnaissance, surveillance, target acquisition, search and rescue, patrolling, real-time monitoring, and mapping, to name a few. To increase the autonomy and the capability of these UAV's and thus to reduce the workload of human operators, typical autonomous UAV's are usually equipped with both a navigation system and a tracking system. The navigation system provides high-rate ownship states (typically ownship inertial position, inertial velocity, and attitude) that are directly used in the autopilot system, and the tracking system provides low-rate target tracking states (typically target relative position and velocity with respect to the ownship). Target states in the global frame can be obtained by adding the ownship states and the target tracking states. The data estimated from this combination of the navigation system and the tracking system provide key information for the design of most UAV guidance laws, control command generation, trajectory generation, and path planning. As a baseline system that estimates ownship states, an integrated navigation system is designed by using an extended Kalman filter (EKF) with sequential measurement updates. In order to effectively fuse various sources of aiding sensor information, the sequential measurement update algorithm is introduced in the design of the integrated navigation system with the objective of being implemented in low-cost autonomous UAV's. Since estimated state accuracy using a low-cost, MEMS-based IMU degrades with time, several absolute (low update rate but bounded error in time) sensors, including the GPS receiver, the magnetometer, and the altimeter, can compensate for time-degrading errors. In this work, the sequential measurement update algorithm in smaller vectors and matrices is capable of providing a convenient framework for fusing the many sources of information in the design of integrated navigation systems. In this framework, several aiding sensor measurements with different size and update rates are easily fused with basic high-rate IMU processing. In order to provide a new mechanism that estimates ownship states, a new nonlinear filtering framework, called the unscented Kalman filter (UKF) with sequential measurement updates, is developed and applied to the design of a new integrated navigation system. The UKF is known to be more accurate and convenient to use with a slightly higher computational cost. This filter provides at least second-order accuracy by approximating Gaussian distributions rather than arbitrary nonlinear functions. This is compared to the first-order accuracy of the well-known EKF based on linearization. In addition, the step of computing the often troublesome Jacobian matrices, always required in the design of an integrated navigation system using the EKF, is eliminated. Furthermore, by employing the concept of sequential measurement updates in the UKF, we can add the advantages of sequential measurement update strategy such as easy compensation of sensor latency, easy fusion of multi-sensors, and easy addition and subtraction of new sensors while maintaining those of the standard UKF such as accurate estimation and removal of Jacobian matrices. Simulation results show better performance of the UKF-based navigation system than the EKF-based system since the UKF-based system is more robust to initial accelerometer and rate gyro biases and more accurate in terms of reducing transient peaks and steady-state errors in ownship state estimation. In order to estimate target tracking states or target kinematics, a new vision-based tracking system is designed by using a UKF in the scenario of three-dimensional air-to-air tracking. The tracking system can estimate not only the target tracking states but also several target characteristics including target size and acceleration. By introducing the UKF, the new vision-based tracking system presents good estimation performance by overcoming the highly nonlinear characteristics of the problem with a relatively simplified formulation. Moreover, the computational step of messy Jacobian matrices involved in the target acceleration dynamics and angular measurements is removed. A new particle filtering framework, called an extended marginalized particle filter (EMPF), is developed and applied to the design of a new vision-based tracking system. In this work, only three position components with vision measurements are solved in particle filtering part by applying Rao-Blackwellization or marginalization approach, and the other dynamics, including the target nonlinear acceleration model, with Gaussian noise are effectively handled by using the UKF. Since vision information can be better represented by probabilistic measurements and the EMPF framework can be easily extended to handle this type of measurements, better performance in estimating target tracking states will be achieved by directly incorporating non-Gaussian, probabilistic vision information as the measurement inputs to the vision-based tracking system in the EMPF framework.

No description available.

This dissertation explores the applicability of RC helicopters as a tool to map wind conditions. This dissertation presents the construction of a robust instrumentation system capable of wireless in-situ measurement and mapping of ship airwake. The presented instrumentation system utilizes an RC helicopter as a carrier platform and uses the helicopter's dynamics for spatial 3D mapping of wind turbulence. The system was tested with a YP676 naval training craft to map ship airwake generated in controlled heading wind conditions. Novel system modeling techniques were developed to estimate the dynamics of an instrumented RC helicopter, in conjunction with onboard sensing, to estimate spatially varying (local) wind conditions. The primary problem addressed in this dissertation is the reliable estimation and separation of pilot induced dynamics from the system measurements, followed by the use of the dynamics residuals/discrepancies to map the wind conditions. This dissertation presents two different modelling approaches to quantify ship airwake using helicopter dynamics. The helicopter systems were characterized using both machine learning and analytical aerodynamic modelling approaches. In the machine learning based approaches, neural networks, along with other models, were trained then assessed in their capability to model dynamics from pilot inputs and other measured helicopter states. The dynamics arising from the wind conditions were fused with the positioning estimates of the helicopter to generate ship airwake maps which were compared against CFD generated airwake patterns. In the analytical modelling based approach, the dynamic response of an RC helicopter to a spatially varying parameterized wind field was modeled using a 30-state nonlinear ordinary differential equation-based dynamic system, while capturing essential elements of the helicopter dynamics. The airwake patterns obtained from both types of approach were compared against anemometrically produced wind maps of turbulent wind conditions artificially generated in a controlled indoor environment. Novel hardware architecture was developed to acquire data critical for the operation and calibration of the proposed system. The mechatronics design of three prototypes of the proposed system were presented and performance evaluated using experimental testing with a modified YP676 naval training vessel in the Chesapeake Bay area. In closing, qualitative analysis of these systems along with potential applications and improvements are discussed to conclude this dissertation. / Ph. D.

The availability of ultra high-frequency (UHF) data on transactions has revolutionised data processing and statistical modelling techniques in finance. The unique characteristics of such data, e.g. discrete structure of price change, unequally spaced time intervals and multiple transactions have introduced new theoretical and computational challenges. In this study, we develop a Bayesian framework for modelling integer-valued variables to capture the fundamental properties of price change. We propose the application of the zero inflated Poisson difference (ZPD) distribution for modelling UHF data and assess the effect of covariates on the behaviour of price change. For this purpose, we present two modelling schemes; the first one is based on the analysis of the data after the market closes for the day and is referred to as off-line data processing. In this case, the Bayesian interpretation and analysis are undertaken using Markov chain Monte Carlo methods. The second modelling scheme introduces the dynamic ZPD model which is implemented through Sequential Monte Carlo methods (also known as particle filters). This procedure enables us to update our inference from data as new transactions take place and is known as online data processing. We apply our models to a set of FTSE100 index changes. Based on the probability integral transform, modified for the case of integer-valued random variables, we show that our models are capable of explaining well the observed distribution of price change. We then apply the deviance information criterion and introduce its sequential version for the purpose of model comparison for off-line and online modelling, respectively. Moreover, in order to add more flexibility to the tails of the ZPD distribution, we introduce the zero inflated generalised Poisson difference distribution and outline its possible application for modelling UHF data.

Este trabalho apresenta um estudo de dados com resposta em intervalos limitados, mais especificamente no intervalo [0,1], como no caso de taxas e proporções. Em diversos casos práticos esta estrutura de dados apresenta uma quantidade não negligenciável de valores extremos (0 e 1) e que modelos usuais não são adequados para sua análise. Para esta situação propomos, por meio de um enfoque Bayesiano, modelos de regressão beta inflacionado de zeros e uns (BIZU) e modelos de regressão Tobit duplamente censurado adaptados nesse intervalo. Técnicas de diagnóstico e qualidade do ajuste também são discutidas. Apresentamos a análise desta estrutura de dados no contexto de série de tempo por meio da abordagem Bayesiana de modelos dinâmicos. Estudos de comportamento e previsão de séries de tempo foram explorados utilizando técnicas de Monte Carlo sequencial, conhecidas como filtro de partículas. Particularidades e competitividade entre as duas classes de modelos também foram discutidas. / This paper presents a study focused on observations in a limited interval , more specifically in [0,1] , such as rate and proportion data. In many practical cases this data structure has a considerable amount of extreme values (0 and 1) and usual classical models are not suitable for this type of data set. We propose two class of regression models to deal with this context: beta inflated of zeros and ones (BIZU) models and Tobit doubly censored models adapted in this interval. Fit quality and diagnostic techniques are also discussed. Time series of proportions are also developed through Bayesian dynamic models. Forecasting and behavioral analysis were explored using sequential Monte Carlo techniques, known as particle filters. Particularities and competitiveness between the two classes of models were also discussed as well.

Nous considérions des modèles à espace d'état où les observations sont multicatégorielles et longitudinales, et l'état est décrit par des modèles du type CHARN. Nous estimons l'état au moyen des récursivités de Kalman généralisées. Celles-ci reposent sur l'application d'une variété de filtres particulaires et de l’algorithme EM. Nos résultats sont appliqués à l'estimation du trait latent en qualité de vie. Ce qui fournit une alternative et une généralisation des méthodes existantes dans la littérature. Ces résultats sont illustrés par des simulations numériques et une application aux données réelles sur la qualité de vie des femmes ayant subi une opération pour cause de cancer du sein / We consider state space models where the observations are multicategorical and longitudinal, and the state is described by CHARN models. We estimate the state by generalized Kalman recursions, which rely on a variety of particle filters and EM algorithm. Our results are applied to estimating the latent trait in quality of life, and this furnishes an alternative and a generalization of existing methods. These results are illustrated by numerical simulations and an application to real data in the quality of life of patients surged for breast cancer

Este trabalho apresenta um estudo de dados com resposta em intervalos limitados, mais especificamente no intervalo [0,1], como no caso de taxas e proporções. Em diversos casos práticos esta estrutura de dados apresenta uma quantidade não negligenciável de valores extremos (0 e 1) e que modelos usuais não são adequados para sua análise. Para esta situação propomos, por meio de um enfoque Bayesiano, modelos de regressão beta inflacionado de zeros e uns (BIZU) e modelos de regressão Tobit duplamente censurado adaptados nesse intervalo. Técnicas de diagnóstico e qualidade do ajuste também são discutidas. Apresentamos a análise desta estrutura de dados no contexto de série de tempo por meio da abordagem Bayesiana de modelos dinâmicos. Estudos de comportamento e previsão de séries de tempo foram explorados utilizando técnicas de Monte Carlo sequencial, conhecidas como filtro de partículas. Particularidades e competitividade entre as duas classes de modelos também foram discutidas. / This paper presents a study focused on observations in a limited interval , more specifically in [0,1] , such as rate and proportion data. In many practical cases this data structure has a considerable amount of extreme values (0 and 1) and usual classical models are not suitable for this type of data set. We propose two class of regression models to deal with this context: beta inflated of zeros and ones (BIZU) models and Tobit doubly censored models adapted in this interval. Fit quality and diagnostic techniques are also discussed. Time series of proportions are also developed through Bayesian dynamic models. Forecasting and behavioral analysis were explored using sequential Monte Carlo techniques, known as particle filters. Particularities and competitiveness between the two classes of models were also discussed as well.