[pt] Esta dissertação investiga as Redes Neurais Bayesianas, que é uma nova abordagem que conjuga o potencial das redes neurais artificiais com a solidez analítica da estatística Bayesiana. Tipicamente, redes neurais convencionais como backpropagation, têm bom desempenho mas apresentam problemas de convergência, na ausência de dados suficientes de treinamento, ou problemas de mínimos locais, que trazem como conseqüência longo tempo de treinamento (esforço computacional) e possibilidades de sobre-treinamento (generalização ruim). Por essas razões, tem-se buscado desenvolver novos algoritmos de aprendizado para redes neurais baseados em princípios que pertencem a outras áreas da ciência como a Estatística, Lógica Nebulosa, Algoritmos Genéticos, etc. Neste sentido, este trabalho estuda e avalia um novo algoritmo de aprendizado baseado na estatística bayesiana, que consiste na utilização do mecanismo de interferência bayesiana no cálculo dos parâmetros (pesos) da rede neural. As principais etapas deste trabalho foram: o estudo das diferenças dos enfoques da estatística clássica e bayesiana sobre o aprendizado das redes neurais; o estudo dos métodos utilizados na inferência bayesiana; a avaliação das redes neurais Bayesianas (RNB) com aplicações Benchmarks; e por último, a avaliação das RNBs com aplicações reais. A diferença entre a estatística clássica e Bayesiana sobre o aprendizado das redes neurais esá na forma em que os parâmetros da rede são calculados. Por exemplo, o princípio de máxima verossimilhança quepertence à estatística clássica, na qual está baseada o algoritmo de backpropagation, se caracteriza por estimar um único vetor de parâmetros da rede neural. Por outro lado, a inferência Bayesiana se caracteriza por calcular uma função de densidade de probabilidade sobre todos os possíveis vetores de parâmetros que a rede neural pode possuir. Os métodos utilizados na inferência Bayesiana para calcular a função de densidade de probabilidade dos parâmetros. Neste trabalho se deu ênfase a dois métodos amplamente utilizados na estatística Bayesiana: o método de aproximação gaussiana e o método de MCMC (Markov Chain Monte Carlo), que mostraram sua efetividade com respeito ao problema da dimensão elevada do vetor de parâmetros. Para avaliar o desempenho destes algoritmos de aprendizado Bayesiano, foram feitos testes em aplicações benchmarks de previsão, classificação e aproximação de uma função. Também foram desenvolvidas aplicações reais de previsão de uma série temporal e carga elétrica e reconhecimento de face onde se avaliou o desempenho destes algoritmos. Além disso, foram feitas comparações entre estes algoritmos de aprendizado Bayesiano com o backpropagation, sistemas neuro fuzzy hierárquicos e outras técnicas estatísticas tais como Box&Jenkins e Holt-Winters. Com este trabalho, verificou-se que entre as vantagens dos algoritmos de aprendizado Bayesiano tem-se: a de minimizar o problema de sobre-treinamento (overfitting); controlar a complexidade do modelo (princípio de Occam’s razor) e ter boa generalização com poucos dados de treinamento. / [en] This dissertation investigates the Bayesianan Neural Networks, which is a new approach that merges the potencial of the artificial neural networks with the robust analytical analysis of the Bayesian Statistic. Typically, theconventional neural networks such as backpropagation, have good performance but presents problems of convergence, when enough data for training is not available, or due to problems of local minimum, which result in long training time and overfitting. For these reasons, researchers are investigating new learning algorithm for neural networks based on principle that belong to other area of science like Statistics, Fuzzy logic, Genetic Algorithms, etc. This dissertation studies and evaluates a new learning algorithm based on the Bayesian Statistics, that consists in the use of the Bayesian mechanical inference to calculate the value of the parameters of neural networks. The main steps of this research are: the study of the difference between the approach of the classical statistics and the approach of the Bayesian statistics regarding the process of learning in neural networks (RNB) with Benchmarks applications; and the evaluation of RNBs with real applications. The main differences between the classical and Bayesian statistics in regard to the learning on neural networks are in the form of calculation of the parameters. For example, the principle of maximum likelihood that belongs to classical statistics, in which the backpropagation algorithms, it is characterized for calculate only on vector of parameters of neural networks. However, the Bayesian inference, it is characterized for calculate a probabilistic density function of the parameters of neural networks are approximations or numerical methods, because the correct analytical treatment is difficult due to the high dimensions of the vector parameter. This dissertation gives especial emphasis to two methods: the Gaussian approximation and the Markov Chain Monte Carlo method (MCMC). To evaluate the performance of these Bayesian learning algorithms, a number of test has been done in application benchmarks of time series forecasting, classification and approximation of functions. Also, have been developed real applications on time serie forecasting of electrical and face recognition. Moreover, comparations have been made between the Bayesian learning algorithms with backpropagation, neuro fuzzy systems and other statistical techniques like a Box&Jenkins and Holt-Winters. This dissertation has shown that the advantages of the Bayesian learning algorithms are the minimization of the overfitting, control of the model complexity (principle of Occam’s razor)and good generalization with a few data for training.

Interest in local food security has increased in the last decade, stemming from concerns surrounding environmental sustainability, agriculture, and community food security. Promotions for consumption of locally produced foods have come from activists, non-governmental organizations, as well as some academic and government research and policy. The goal of this thesis is to develop, map, and model an index of self-sufficiency in the province of British Columbia. To meet this goal, I develop estimates for food production at the local scale by integrating federally gathered agricultural land use and yield data from the Agricultural Census and various surveys. Second, I construct population-level food consumption estimates based on provincial nutrition survey and regional demographics. Third, I construct a self-sufficiency index for each Local Health Area in the province, and develop a predictive model in a Bayesian autoregressive framework. I find that local scale comparable estimates of food production and food consumption can be constructed through data integration, and both datasets exhibit considerable spatial variability throughout the province. The predictive model allows for estimation of regional scale self-sufficiency without reliance on land use or nutrition data and stabilizes mapping of our raw index through neighborhood-based spatial smoothing. The methods developed will be a useful tool for researchers and government officials interested in agriculture, nutrition, and food security, as well as a first step towards more advanced modeling of current local food self-sufficiency and future potential. / Graduate

In the era of "big data", scalable statistical inference is necessary to learn from new and growing sources of quantitative information. However, many commercial and scientific applications also require models to be interpretable to end users in order to generate actionable insights about quantities of interest. We present three case studies of Bayesian hierarchical models that improve the interpretability of existing models while also maintaining or improving the efficiency of inference. The first paper is an application to online advertising that presents an augmented regression model interpretable in terms of the amount of revenue a customer is expected to generate over his or her entire relationship with the company---even if complete histories are never observed. The resulting Poisson Process Regression employs a marginal inference strategy that avoids specifying customer-level latent variables used in previous work that complicate inference and interpretability. The second and third papers are applications to the analysis of text data that propose improved summaries of topic components discovered by these mixture models. While the current practice is to summarize topics in terms of their most frequent words, we show significantly greater interpretability in online experiments with human evaluators by using words that are also relatively exclusive to the topic of interest. In the process we develop a new class of topic models that directly regularize the differential usage of words across topics in order to produce stable estimates of the combined frequency-exclusivity metric as well as proposing efficient and parallelizable MCMC inference strategies. / Statistics

Time series models are used to characterise uncertainty in many real-world dynamical phenomena. A time series model typically contains a static variable, called parameter, which parametrizes the joint law of the random variables involved in the definition of the model. When a time series model is to be fitted to some sequentially observed data, it is essential to decide on the value of the parameter that describes the data best, a procedure generally called parameter estimation. This thesis comprises novel contributions to the methodology on parameter estimation in time series models. Our primary interest is online estimation, although batch estimation is also considered. The developed methods are based on batch and online versions of expectation-maximisation (EM) and gradient ascent, two widely popular algorithms for maximum likelihood estimation (MLE). In the last two decades, the range of statistical models where parameter estimation can be performed has been significantly extended with the development of Monte Carlo methods. We provide contribution to the field in a similar manner, namely by combining EM and gradient ascent algorithms with sequential Monte Carlo (SMC) techniques. The time series models we investigate are widely used in statistical and engineering applications. The original work of this thesis is organised in Chapters 4 to 7. Chapter 4 contains an online EM algorithm using SMC for MLE in changepoint models, which are widely used to model heterogeneity in sequential data. In Chapter 5, we present batch and online EM algorithms using SMC for MLE in linear Gaussian multiple target tracking models. Chapter 6 contains a novel methodology for implementing MLE in a hidden Markov model having intractable probability densities for its observations. Finally, in Chapter 7 we formulate the nonnegative matrix factorisation problem as MLE in a specific hidden Markov model and propose online EM algorithms using SMC to perform MLE.

In this thesis I carry out an encompassing analysis of the problem of measurement error in retrospectively collected work histories using data from the “Longitudinal Study of the Unemployed”. This dataset has the unique feature of linking survey responses to a retrospective question on work status to administrative data from the Swedish Register of Unemployment. Under the assumption that the register data is a gold standard I explore three research questions: i) what is the prevalence of and the reasons for measurement error in retrospective reports of unemployment; ii) what are the consequences of using such survey data subject to measurement error in event history analysis; and iii) what are the most effective statistical methods to adjust for such measurement error. Regarding the first question I find substantial measurement error in retrospective reports of unemployment, e.g. only 54% of the subjects studied managed to report the correct number of spells of unemployment experienced in the year prior to the interview. Some reasons behind this problem are clear, e.g. the longer the recall period the higher the prevalence of measurement error. However, some others depend on how measurement error is defined, e.g. women were associated with a higher probability of misclassifying spells of unemployment but not with misdating them. To answer the second question I compare different event history models using duration data from the survey and the register as their response variable. Here I find that the impact of measurement error is very large, attenuating regression estimates by about 90% of their true value, and this impact is fairly consistent regardless of the type of event history model used. In the third part of the analysis I implement different adjustment methods and compare their effectiveness. Here I note how standard methods based on strong assumptions such as SIMEX or Regression Calibration are incapable of dealing with the complexity of the measurement process under analysis. More positive results are obtained through the implementation of ad hoc Bayesian adjustments capable of accounting for the different patterns of measurement error using a mixture model.

Soil liquefaction, or loss of strength due to excess pore water pressures generated during dynamic loading, is a main cause of damage during earthquakes. When a soil liquefies (referred to as triggering), it may lose its ability to support overlying structures, deform vertically or laterally, or cause buoyant uplift of buried utilities. Empirical liquefaction models, used to predict liquefaction potential based upon in-situ soil index property measurements and anticipated level of seismic loading, are the standard of practice for assessing liquefaction triggering. However, many current models do not incorporate predictor variable uncertainty or do so in a limited fashion. Additionally, past model creation and validation lacks the same rigor found in predictive modeling in other fields. This study examines the details of creating and validating an empirical liquefaction model, using the existing worldwide cone penetration test liquefaction database. Our study implements a logistic regression within a Bayesian measurement error framework to incorporate uncertainty in predictor variables and allow for a probabilistic interpretation of model parameters. Our model is built using a hierarchal approach account for intra-event correlation in loading variables and differences in event sample sizes that mirrors the random/mixed effects models used in ground motion prediction equation development. The model is tested using an independent set of case histories from recent New Zealand earthquakes, and performance metrics are reported. We found that a Bayesian measurement error model considering two predictor variables, qc,1 and CSR, decreases model uncertainty while maintaining predictive utility for new data. Two forms of model uncertainty were considered – the spread of probabilities predicted by mean values of regression coefficients (apparent uncertainty) and the standard deviations of the predictive distributions from fully probabilistic inference. Additionally, we found models considering friction ratio as a predictor variable performed worse than the two variable case and will require more data or informative priors to be adequately estimated.

Un cadre statistique bayésien a été créé dans le cadre de cette thèse pour le développement de modèles cliniques basés sur une approche d’apprentissage continu dans laquelle de nouvelles données peuvent être ajoutées. L’objectif des modèles est de prévoir les effets de la radiothérapie à partir de preuves cliniques. Des concepts d’apprentissage machine ont été utilisés pour résoudre le cadre bayésien. Les modèles développés concernent un cancer du cerveau agressif appelé glioblastome. Les données médicales comprennent une base de données d’environ 90 patients souffrant de glioblastome ; la base de données contient des images médicales et des entrées de données telles que l’âge, le sexe, etc. Des modèles de prévision neurologique ont été construits pour illustrer le type de modèles qui sont obtenus avec la méthodologie. Des modèles de récidive du glioblastome, sous la forme de modèles linéaires généralisés (GLM) et de modèles d’arbres de décision, ont été développés pour explorer la possibilité de prédire l’emplacement de la récidive à l’aide de l’imagerie préradiothérapie. Faute d’une prédiction suffisamment forte obtenue par les modèles arborescents, nous avons décidé de développer des outils de représentation visuelle. Ces outils permettent d’observer directement les valeurs d’intensité des images médicales concernant les lieux de récidive et de non-récurrence. Dans l’ensemble, le cadre élaboré pour la modélisation des données cliniques en radiothérapie fournit une base solide pour l’élaboration de modèles plus complexes. / A Bayesian statistics framework was created in this thesis work for developing clinical based models in a continuous learning approach in which new data can be added. The objective of the models is to forecast radiation therapy effects based on clinical evidence. Machine learning concepts were used for solving the Bayesian framework. The models developed concern an aggressive brain cancer called glioblastoma. The medical data comprises a database of about 90 patients suffering glioblastoma; the database contains medical images and data entries such as age, gender, etc. Neurologic grade predictions models were constructed for illustrating the type of models that can be build with the methodology. Glioblastoma recurrence models, in the form of Generalized Linear Models (GLM) and decision tree models, were developed to explore the possibility of predicting the recurrence location using pre-radiation treatment imaging. Following, due to the lack of a sufficiently strong prediction obtained by the tree models, we decided to develop visual representation tools to directly observe the medical image intensity values concerning the recurrence and non-recurrence locations. Overall, the framework developed for modeling of radiation therapy clinical data provides a solid foundation for more complex models to be developed.

Anomaly detection is a relevant problem in the field of Mechanical Engineering, because the analysis of mechanical systems often relies on identifying deviations from what is considered "normal". The mechanical sciences are represented by a heterogeneous collection of data types: some systems may be highly dimensional, may contain exclusively spatial or temporal data, may be spatiotemporally linked, or may be non-deterministic and best described probabilistically. Given the broad range of data types in this field, it is not possible to propose a single processing method that will be appropriate, or even usable, for all data types. This has led to human observation remaining a common, albeit costly and inefficient, approach to detecting anomalous signals or patterns in mechanical data. The advantages of automated anomaly detection in mechanical systems include reduced monitoring costs, increased reliability of fault detection, and improved safety for users and operators. This dissertation proposes a hierarchical framework for anomaly detection through machine learning, and applies it to three distinct and heterogeneous data types: state-based data, parameter-driven data, and spatiotemporal sensor network data. In time-series data, anomaly detection results were robust in synthetic data generated using multiple simulation algorithms, as well as experimental data from rolling element bearings, with highly accurate detection rates (>99% detection, <1% false alarm). Significant developments were shown in parameter-driven data by reducing the sample sizes necessary for analysis, as well as reducing the time required for computation. The event-space model extends previous work into a geospatial sensor network and demonstrates applications of this type of event modeling at various timescales, and compares the model to results obtained using other approaches. Each data type is processed in a unique way relative to the others, but all are fitted to the same hierarchical structure for system modeling. This hierarchical model is the key development proposed by this dissertation, and makes both novel and significant contributions to the fields of mechanical analysis and data processing. This work demonstrates the effectiveness of the developed approaches, details how they differ from other relevant industry standard methods, and concludes with a proposal for additional research into other data types. / Ph. D.

No description available.

Computer models have become useful research tools in many disciplines. In many cases a researcher has access to data from a computer simulator and from a physical system. This research discusses Bayesian models that allow for the estimation of the discrepancy between the two data sources. We fit two models to data in the field of electrical engineering. Using this data we illustrate ways of modeling both a deterministic and a stochastic simulator when specific parametric assumptions can be made about the discrepancy term.