Probabilistic models have been successfully applied for a wide variety of problems, such as but not limited to information retrieval, computer vision, bio-informatics and speech processing. Probabilistic models allow us to encode our assumptions about the data in an elegant fashion and enable us to perform machine learning tasks such as classification and clustering in a principled manner. Probabilistic models for bio-acoustic data help in identifying interesting patterns in the data (for instance, the species-specific vocabulary), as well as species identification (classification) in recordings where the label is not available. The focus of this thesis is to develop efficient inference techniques for existing models, as well as develop probabilistic models tailored to bioacoustic data. First, we develop inference algorithms for the supervised latent Dirichlet allocation (LDA) model. We present collapsed variational Bayes, collapsed Gibbs sampling and maximum-a-posteriori (MAP) inference for parameter estimation and classification in supervised LDA. We provide an empirical evaluation of the trade-off between computational complexity and classification performance of the inference methods for supervised LDA, on audio classification (species identification in this context)as well as image classification and document classification tasks. Next, we present novel probabilistic models for bird sound recordings, that can capture temporal structure at different hierarchical levels, and model additional information such as the duration and frequency of vocalizations. We present a non-parametric density estimation technique for parameter estimation and show that the MAP classifier for our models can be interpreted as a weighted nearest neighbor classifier. We provide an experimental comparison between the proposed models and a support vector machine based approach, using bird sound recordings from the Cornell Macaulay library. / Graduation date: 2011 / Access restricted to the OSU Community at author's request from Dec. 30, 2010 - Dec. 30, 2011

Nowadays, probabilistic models are playing a significant role in various areas in- cluding machine learning, artificial intelligence and cognitive science, etc. How- ever, as those models are becoming more and more complex, it shows that the corresponding programs are really hard to maintain and reuse as well. Meanwhile, the current tools are not feasible enough to enable probabilistic modeling and ma- chine learning to be accessible to the working programmer, who has sufficient do- main expertise, but perhaps not enough expertise in probability theory or machine learning. Probabilistic programming is one possible way to solve this. Indeed, probabilistic programming languages are powerful tools to specify probabilistic models directly in terms of a computer programs. While programmers writes normal procedures, everything will be automatically translated into statistical distributions and then users can do inferences upon them. This project aims at exploring and implementing a probabilistic programming language, for which we name as Shelang. We use Scheme, a dialect of Lisp lan- guage which is originated from λ-Calculus, to implement a embedded probabilis- tic programming language. This paper mainly discusses about the design, algo- rithms, details of this implementation and several usages of Shelang and make a conclusion in the end.

No description available.

Actuellement, les équipements employés dans les milieux industriels sont de plus en plus complexes. Ils exigent une maintenance accrue afin de garantir un niveau de service optimal en termes de fiabilité et de disponibilité. Par ailleurs, souvent cette garantie d'optimalité a un coût très élevé, ce qui est contraignant. Face à ces exigences la gestion de la maintenance des équipements est désormais un enjeu de taille : rechercher une politique de maintenance réalisant un compromis acceptable entre la disponibilité et les coûts associés à l'entretien du système. Les travaux de cette thèse partent par ailleurs du constat que dans plusieurs applications de l'industrie, le besoin de stratégies de maintenance assurant à la fois une sécurité optimale et une rentabilité maximale demeure de plus en plus croissant conduisant à se référer non seulement à l'expérience des experts, mais aussi aux résultats numériques obtenus via la résolution des problèmes d'optimisation. La résolution de cette problématique nécessite au préalable la modélisation de l'évolution des comportements des états des composants constituant le système, i.e, connaître les mécanismes de dégradation des composants. Disposant d'un tel modèle, une stratégie de maintenance est appliquée au système. Néanmoins, l'élaboration d'une telle stratégie réalisant un compromis entre toutes ces exigences représente un verrou scientifique et technique majeur. Dans ce contexte, l'optimisation de la maintenance s'impose pour atteindre les objectifs prescrits avec des coûts optimaux. Dans les applications industrielles réelles, les problèmes d'optimisation sont souvent de grande dimension faisant intervenir plusieurs paramètres. Par conséquent, les métaheuristiques s’avèrent une approche intéressante dans la mesure où d'une part, elles sacrifient la complétude de la résolution au profit de l'efficacité et du temps de calcul et d'autre part elles s'appliquent à un très large panel de problèmes.Dans son objectif de proposer une démarche de résolution d'un problème d'optimisation de la maintenance préventive, cette thèse fournit une méthodologie de résolution du problème d'optimisation des politiques de maintenance préventive systématique appliquée dans le domaine ferroviaire à la prévention des ruptures de rails. Le raisonnement de cette méthodologie s'organise autour de trois étapes principales : 1. Modélisation de l'évolution des comportements des états des composants constituant le système, i.e, connaître les mécanismes de dégradation des composants et formalisation des opérations de maintenance. 2. Formalisation d'un modèle d'évaluation de politiques de maintenance tenant compte aussi bien du facteur sûreté de fonctionnement du système que du facteur économique conséquent aux procédures de gestion de la maintenance (coûts de réparation, de diagnostic, d'indisponibilité). 3. Optimisation des paramètres de configuration des politiques de maintenance préventive systématique afin d'optimiser un ou plusieurs critères. Ces critères sont définis sur la base du modèle d'évaluation des politiques de maintenance proposé dans l'étape précédente / At present, equipments used on the industrial circles are more and more complex. They require a maintenance increased to guarantee a level of optimal service in terms of reliability and availability. Besides, often this guarantee of optimalité has a very high cost, what is binding. In the face of these requirements the management of the maintenance of equipments is from now on a stake in size: look for a politics of maintenance realizing an acceptable compromise between the availability and the costs associated to the maintenance of the system. The works of this thesis leave besides the report that in several applications of the industry, the need for strategies of maintenance assuring(insuring) at the same time an optimal safety and a maximal profitability lives furthermore there

Structure is one of the fundamentals of music, yet the complexity arising from the vast number of possible variations of musical elements such as rhythm, melody, harmony, key, texture and form, along with their combinations, makes music modelling a particularly challenging task for machine learning. The research presented in this thesis focuses on the problem of learning a generative model for melody directly from musical sequences belonging to the same genre. Our goal is to develop probabilistic models that can automatically capture the complex statistical dependencies evident in music without the need to incorporate significant domain-specifc knowledge. At all stages we avoid making assumptions explicit to music and consider models that can can be readily applied in different music genres and can easily be adapted for other sequential data domains. We develop the Dirichlet Variable-Length Markov Model (Dirichlet-VMM), a Bayesian formulation of the Variable-Length Markov Model (VMM), where smoothing is performed in a systematic probabilistic manner. The model is a general-purpose, dictionary-based predictor with a formal smoothing technique and is shown to perform significantly better than the standard VMM in melody modelling. Motivated by the ability of the Restricted Boltzmann Machine (RBM) to extract high quality latent features in an unsupervised manner, we next develop the Time-Convolutional Restricted Boltzmann Machine (TC-RBM), a novel adaptation of the Convolutional RBM for modelling sequential data. We show that the TC-RBM learns descriptive musical features such as chords, octaves and typical melody movement patterns. To deal with the non-stationarity of music, we develop the Variable-gram Topic model, which employs the Dirichlet-VMM for the parametrisation of the topic distributions. The Dirichlet-VMM models the local temporal structure, while the latent topics represent di erent music regimes. The model does not make any assumptions explicit to music, but it is particularly suitable in this context, as it couples the latent topic formalism with an expressive model of contextual information.

A key goal for humans and artificial intelligence systems is to develop an accurate and unified picture of the outside world based on the data from any sense(s) that may be available. The availability of multiple senses presents the perceptual system with new opportunities to fulfil this goal, but exploiting these opportunities first requires the solution of two related tasks. The first is how to make the best use of any redundant information from the sensors to produce the most accurate percept of the state of the world. The second is how to interpret the relationship between observations in each modality; for example, the correspondence problem of whether or not they originate from the same source. This thesis investigates these questions using ideal Bayesian observers as the underlying theoretical approach. In particular, the latter correspondence task is treated as a problem of Bayesian model selection or structure inference in Bayesian networks. This approach provides a unified and principled way of representing and understanding the perceptual problems faced by humans and machines and their commonality. In the domain of machine intelligence, we exploit the developed theory for practical benefit, developing a model to represent audio-visual correlations. Unsupervised learning in this model provides automatic calibration and user appearance learning, without human intervention. Inference in the model involves explicit reasoning about the association between latent sources and observations. This provides audio-visual tracking through occlusion with improved accuracy compared to standard techniques. It also provides detection, verification and speech segmentation, ultimately allowing the machine to understand ``who said what, where?'' in multi-party conversations. In the domain of human neuroscience, we show how a variety of recent results in multimodal perception can be understood as the consequence of probabilistic reasoning about the causal structure of multimodal observations. We show this for a localisation task in audio-visual psychophysics, which is very similar to the task solved by our machine learning system. We also use the same theory to understand results from experiments in the completely different paradigm of oddity detection using visual and haptic modalities. These results begin to suggest that the human perceptual system performs -- or at least approximates -- sophisticated probabilistic reasoning about the causal structure of observations under the hood.

Probabilistic machine learning and circular statistics—the branch of statistics concerned with data as angles and directions—are two research communities that have grown mostly in isolation from one another. On the one hand, probabilistic machine learning community has developed powerful frameworks for problems whose data lives on Euclidean spaces, such as Gaussian Processes, but have generally neglected other topologies studied by circular statistics. On the other hand, the approximate inference frameworks from probabilistic machine learning have only recently started to the circular statistics landscape. This thesis intends to redress the gap between these two fields by contributing to both fields with models and approximate inference algorithms. In particular, we introduce the multivariate Generalised von Mises distribution (mGvM), which allows the use of kernels in circular statistics akin to Gaussian Processes, and an augmented representation. These models account for a vast number of applications comprising both latent variable modelling and regression of circular data. Then, we propose methods to conduct approximate inference on these models. In particular, we investigate the use of Variational Inference, Expectation Propagation and Markov chain Monte Carlo methods. The variational inference route taken was a mean field approach to efficiently leverage the mGvM tractable conditionals and create a baseline for comparison with other methods. Then, an Expectation Propagation approach is presented drawing on the Expectation Consistent Framework for Ising models and connecting the approximations used to the augmented model presented. In the final MCMC chapter, efficient Gibbs and Hamiltonian Monte Carlo samplers are derived for the mGvM and the augmented model.

Made available in DSpace on 2017-07-10T19:25:05Z (GMT). No. of bitstreams: 1 Calil Abumanssur.pdf: 713592 bytes, checksum: 8bf0b3044a8980a927948637dc210303 (MD5) Previous issue date: 2006-07-25 / A set of measurements of daily average temperatures of 22 agrometeorológicas stations of the Agronomic Institute of the Paraná-IAPAR in operation, were compiled originating monthly storms series matrices of temperature that permitted to calculate to evapotranspiration by the Method of Camargo from which were obtained values of the evapotranspiration month to month for each one of the distinct stations. It applied the models of the gama distribution functions, log-normal and generalized of extreme value was utilizing the results of the monthly evapotranspiration for calculate the probable value with 75% of occurrence probability. For verification of adjustment of the curves of variation of the functions distribution of the models log-normal end gamma the method of Kolmogorov-Smirnov was used, for the generalized model of extreme values the verification of the adjustment was used of the method of Wang. As they turned out was observed the monthly variation of the values of evapotranspiration to the long one of the months in the State, where the tendency of regions more to the north will have the biggest values estimated of sheet of evapotranspiration and in the stations situated in areas of bigger relative altitudes will present sheets values minors. As regards the models range, log-normal and generalized extreme value, utilized for estimate of the evapotranspiration potential, if adjusted satisfactorily. With the values of each month, considering the probability of 75% of occurrence, in each one of the stations, was mounted the map of the isolines that represent the variability of the evapotranspiration potential, according to model gama, on the cartographic base of the State of the Paraná/Brazil. / Um conjunto de medições de temperaturas médias diárias de 22 estações agrometeorológicas do Instituto Agronômico do Paraná - IAPAR em operação, foram compiladas originando matrizes de séries temporais mensais de temperatura que permitiram calcular a evapotranspiração pelo Método de Camargo, a partir do qual foram obtidos valores da evapotranspiração mês a mês para cada uma das estações distintas. Aplicando-se os modelos das funções de distribuição gama, log-normal e generalizada de valores extremos, foram utilizados os resultados da evapotranspiração mensal para calcular o valor provável, com 75% de probabilidade de ocorrência. Para verificação de ajuste das curvas de variação das funções distribuição dos modelos gama e log-normal foi utilizado o método de Kolmogorov-Smirnov, para o modelo generalizado de valores extremos a verificação do ajuste utilizou-se do método de Wang. Como resultado, foi observada a variação mensal dos valores de evapotranspiração no Estado, ao longo dos meses. Verificou-se a tendência de regiões mais ao norte apresentarem os maiores valores estimados de lâmina de evapotranspiração e nas estações situadas em áreas de maiores altitudes relativas apresentarem menores valores de lâminas. Quanto aos modelos gama, log-normal e generalizada de valores extremos, utilizados para estimativa da evapotranspiração potencial, ajustaram se satisfatoriamente. Com os valores de cada mês, considerando a probabilidade de 75% de ocorrência, em cada uma das estações, foi montado o mapa das isolinhas que representa a variabilidade da evapotranspiração potencial, segundo o modelo gama, sobre a base cartográfica do estado do Paraná.

Made available in DSpace on 2017-05-12T14:48:27Z (GMT). No. of bitstreams: 1 Calil Abumanssur.pdf: 713592 bytes, checksum: 8bf0b3044a8980a927948637dc210303 (MD5) Previous issue date: 2006-07-25 / A set of measurements of daily average temperatures of 22 agrometeorológicas stations of the Agronomic Institute of the Paraná-IAPAR in operation, were compiled originating monthly storms series matrices of temperature that permitted to calculate to evapotranspiration by the Method of Camargo from which were obtained values of the evapotranspiration month to month for each one of the distinct stations. It applied the models of the gama distribution functions, log-normal and generalized of extreme value was utilizing the results of the monthly evapotranspiration for calculate the probable value with 75% of occurrence probability. For verification of adjustment of the curves of variation of the functions distribution of the models log-normal end gamma the method of Kolmogorov-Smirnov was used, for the generalized model of extreme values the verification of the adjustment was used of the method of Wang. As they turned out was observed the monthly variation of the values of evapotranspiration to the long one of the months in the State, where the tendency of regions more to the north will have the biggest values estimated of sheet of evapotranspiration and in the stations situated in areas of bigger relative altitudes will present sheets values minors. As regards the models range, log-normal and generalized extreme value, utilized for estimate of the evapotranspiration potential, if adjusted satisfactorily. With the values of each month, considering the probability of 75% of occurrence, in each one of the stations, was mounted the map of the isolines that represent the variability of the evapotranspiration potential, according to model gama, on the cartographic base of the State of the Paraná/Brazil. / Um conjunto de medições de temperaturas médias diárias de 22 estações agrometeorológicas do Instituto Agronômico do Paraná - IAPAR em operação, foram compiladas originando matrizes de séries temporais mensais de temperatura que permitiram calcular a evapotranspiração pelo Método de Camargo, a partir do qual foram obtidos valores da evapotranspiração mês a mês para cada uma das estações distintas. Aplicando-se os modelos das funções de distribuição gama, log-normal e generalizada de valores extremos, foram utilizados os resultados da evapotranspiração mensal para calcular o valor provável, com 75% de probabilidade de ocorrência. Para verificação de ajuste das curvas de variação das funções distribuição dos modelos gama e log-normal foi utilizado o método de Kolmogorov-Smirnov, para o modelo generalizado de valores extremos a verificação do ajuste utilizou-se do método de Wang. Como resultado, foi observada a variação mensal dos valores de evapotranspiração no Estado, ao longo dos meses. Verificou-se a tendência de regiões mais ao norte apresentarem os maiores valores estimados de lâmina de evapotranspiração e nas estações situadas em áreas de maiores altitudes relativas apresentarem menores valores de lâminas. Quanto aos modelos gama, log-normal e generalizada de valores extremos, utilizados para estimativa da evapotranspiração potencial, ajustaram se satisfatoriamente. Com os valores de cada mês, considerando a probabilidade de 75% de ocorrência, em cada uma das estações, foi montado o mapa das isolinhas que representa a variabilidade da evapotranspiração potencial, segundo o modelo gama, sobre a base cartográfica do estado do Paraná.

Esta tesis está dirigida principalmente a investigadores interesados en la aplicación de técnicas de minera de datos en Meteorología y otras ciencias medioambientales afines. De forma genérica, trata de la modelización probabilística de sistemas definidos por muchas variables, cuyas relaciones de dependencia son inferidas a partir de un conjunto representativo de datos. La idea es resolver algunos problemas prácticos relacionados con el diagnóstico y la predicción probabilística local en Meteorología, considerando el problema de la coherencia espacial. En concreto, el eje central de esta tesis ha sido el desarrollo de redes Bayesianas, para su aplicación en la predicción probabilística local. / This thesis is mainly oriented to researchers interested in the data mining techniques applied to Meteorology and other related environmental sciences. It uses probabilistic models to describe systems defined by many variables whose dependencies have to be inferred from a set of representative data. The main purpose is solve practical problems related to the diagnosis and probabilistic local forecasting Meteorology, considering the problem of spatial coherence. Specifically, the focus of this thesis has been the development of Bayesian networks to be applied in the local probabilistic forecasting.