In the last decades, the explosion of data from quantitative techniques has revolutionised our understanding of biological processes. In this scenario, advanced statistical methods and algorithms are becoming fundamental to decipher the dynamics of biochemical mechanisms such those involved in the regulation of gene expression. Here we develop mechanistic models and approximate inference techniques to reverse engineer the dynamics of gene regulation, from mRNA and/or protein time series data. We start from an existent variational framework for statistical inference in transcriptional networks. The framework is based on a continuous-time description of the mRNA dynamics in terms of stochastic differential equations, which are governed by latent switching variables representing the on/off activity of regulating transcription factors. The main contributions of this work are the following. We speeded-up the variational inference algorithm by developing a method to compute a posterior approximate distribution over the latent variables using a constrained optimisation algorithm. In addition to computational benefits, this method enabled the extension to statistical inference in networks with a combinatorial model of regulation. A limitation of this framework is the fact that inference is possible only in transcriptional networks with a single-layer architecture (where a single or couples of transcription factors regulate directly an arbitrary number of target genes). The second main contribution in this work is the extension of the inference framework to hierarchical structures, such as feed-forward loop. In the last contribution we define a general structure for transcription-translation networks. This work is important since it provides a general statistical framework to model complex dynamics in gene regulatory networks. The framework is modular and scalable to realistically large systems with general architecture, thus representing a valuable alternative to traditional differential equation models. All models are embedded in a Bayesian framework; inference is performed using a variational approach and compared to exact inference where possible. We apply the models to the study of different biological systems, from the metabolism in E. coli to the circadian clock in the picoalga O. tauri.

The activity of cortical neurons in awake brains changes dynamically as a function of the behavioural and attentional state. The primary motor cortex (M1) plays a central role in regulating complex motor behaviors. Despite a growing knowledge on its connectivity and spiking pattern, little is known about intra-cellular mechanism and rhythms underlying motor-command generation. In the last decade, whole-cell recordings in awake animals has become a powerful tool for characterising both sub-and supra-threshold activity during behaviour. Seminal in vivo studies have shown that changes in input structure and sub-threshold regime determine spike output during behaviour (input-output transformations). In this thesis I make use of computational and experimental techniques to better understand (i) how the brain regulates the sub-threshold activity of the neurons during movement and (ii) how this reflects in their input-output transformation properties. In the first part of this work I present a novel probabilistic technique to infer input statistics from in-vivo voltage-clamp traces. This approach, based on Bayesian belief networks, outperforms current methods and allows an estimation of synaptic input (i) kinetic properties, (ii) frequency, and (iii) weight distribution. I first validate the model on simulated data, thus I apply it to voltage-clamp recordings of cerebellar interneurons in awake mice. I found that synaptic weight distributions have long tails, which on average do not change during movement. Interestingly, the increase in synaptic current observed during movement is a consequence of the increase in input frequency only. In the second part, I study how the brain regulates the activity of pyramidal neurons in the M1 of awake mice during movement. I performed whole-cell recordings of pyramidal neurons in layer 5B (L5B), which represent one of the main descending output channels from motor cortex. I found that slow large-amplitude membrane potential fluctuations, typical of quiet periods, were suppressed in all L5B pyramidal neurons during movement, which by itself reduced membrane potential (Vm) variability, input sensitivity and output firing rates. However, a sub-population of L5B neurons ( 50%) concurrently experienced an increase in excitatory drive that depolarized mean Vm, enhanced input sensitivity and elevated firing rates. Thus, movement-related bidirectional modulation in L5B neurons is mediated by two opposing mechanisms: 1) a global reduction in network driven Vm variability and 2) a coincident, targeted increase in excitatory drive to a subpopulation of L5B neurons.

This paper discusses characteristics of standard conjugate priors and their induced posteriors in Bayesian inference for von Mises-Fisher distributions, using either the canonical natural exponential family or the more commonly employed polar coordinate parameterizations. We analyze when standard conjugate priors as well as posteriors are proper, and investigate the Jeffreys prior for the von Mises-Fisher family. Finally, we characterize the proper distributions in the standard conjugate family of the (matrixvalued) von Mises-Fisher distributions on Stiefel manifolds.

O uso de fronteira de produ¸c˜ ao estoc´ astica com m´ ultiplos produtos tem despertado um interesse especial em ´areas da economia que defrontam-se com o problema de quantificar a eficiˆencia t´ecnica de firmas. Na estat´ýstica cl´ assica, quando se defronta com firmas que possuem v´arios produtos, as fun¸c˜ oes custo ou demanda s˜ ao mais utilizadas para calcular essa eficiˆencia, mas isso requer uma quantidade maior de informa¸c˜ oes sobre os dados, al´em das quantidades de insumos e produtos, tamb´em s˜ ao necess´ arios seus pre¸cos e custos. Quando existem apenas informa¸c˜ oes sobre os insumos (x) e os produtos (y) h´a a necessidade de se trabalhar com a fun¸c˜ ao de produ¸c˜ ao e a inexistˆencia de estat´ýsticas suficientes para alguns parˆ ametros tornam a an´alise d´ýficil. A abordagem Bayesiana pode se tornar uma ferramenta muito ´ util para esse caso, pois ´e poss´ývel obter uma amostra da distribui¸ c˜ ao de probabilidade dos parˆ ametros do modelo, possibilitando a obten¸c˜ ao de resumos de interesse. Para obter as amostras dessas distribui¸ c˜ oes m´etodos Monte Carlo com cadeias de Markov, tais como, amostrador de Gibbs, Metropolis-Hastings e "Slice sampling" s˜ ao utilizados. / The use of stochastic production frontier with multiple-outputs has been waking up a special interest in areas of the economy that are confronted with the problem of quantifying the technical efficiency of firms. In the classic statistics, when it is confronted with firms that possess several outputs, cost or profit functions are more used to calculate that efficiency, but that requests an amount larger of information about data set, besides the amounts of inputs and outputs, are also necessary your prices and costs. When just exist information on inputs (x) and outputs (y) there is need to work with the production function and the lack of enough statistics for some parameters turn the difficult analysis. Bayesian approach can become a useful tool for that case, because is possible to obtain a sample of the distribution of probability of the parameters of the model, making possible the obtaining of summaries of interest. To obtain samples of those distributions methods Markov chains Monte Carlo, that is, Gibbs sampling, Metropolis-Hastings and Slice sampling are used.

Este trabalho tem o objetivo de divulgar a metodologia de ponderação de modelos ou Bayesian Model Averaging (BMA) entre os pesquisadores da área agronômica e discutir suas vantagens e limitações. Com o BMA é possível combinar resultados de diferentes modelos acerca de determinada quantidade de interesse, com isso, o BMA apresenta-se como sendo uma metodologia alternativa de análise de dados frente os usuais métodos de seleção de modelos tais como o Coeficiente de Determinação Múltipla (R2 ), Coeficiente de Determinação Múltipla Ajustado (R2), Estatística de Mallows ( Cp) e Soma de Quadrados de Predição (PRESS). Vários trabalhos foram, recentemente, realizados com o objetivo de comparar o desempenho do BMA em relação aos métodos de seleção de modelos, porém, há ainda muitas situações para serem exploradas até que se possa chegar a uma conclusão geral acerca desta metodologia. Neste trabalho, o BMA foi aplicado a um conjunto de dados proveniente de um experimento agronômico. A seguir, o desempenho preditivo do BMA foi comparado com o desempenho dos métodos de seleção acima citados por meio de um estudo de simulação variando o grau de multicolinearidade e o tamanho amostral. Em cada uma dessas situações, foram utilizadas 1000 amostras geradas a partir de medidas descritivas de conjuntos de dados reais da área agronômica. O desempenho preditivo das metodologias em comparação foi medido pelo Logaritmo do Escore Preditivo (LEP). Os resultados empíricos obtidos indicaram que o BMA apresenta desempenho semelhante aos métodos usuais de seleção de modelos nas situações de multicolinearidade exploradas neste trabalho. / The objective of this work was divulge to Bayesian Model Averaging (BMA) between the researchers of the agronomy area and discuss its advantages and limitations. With the BMA is possible combine results of difeerent models about determined quantity of interest, with that, the BMA presents as being a metodology alternative of data analysis front the usual models selection approaches, for example the Coefficient of Multiple Determination (R2), Coefficient of Multiple Determination Adjusted (R2), Mallows (Cp Statistics) and Prediction Error Sum Squares (PRESS). Several works recently were carried out with the objective of compare the performance of the BMA regarding the approaches of models selection, however, there is still many situations for will be exploited to that can arrive to a general conclusion about this metodology. In this work, the BMA was applied to data originating from an agronomy experiment. It follow, the predictive performance of the BMA was compared with the performance of the approaches of selection above cited by means of a study of simulation varying the degree of multicollinearity, measured by the number of condition of the matrix standardized X'X and the number of observations in the sample. In each one of those situations, were utilized 1000 samples generated from the descriptive information of agronomy data. The predictive performance of the metodologies in comparison was measured by the Logarithm of the Score Predictive (LEP). The empirical results obtained indicated that the BMA presents similar performance to the usual approaches of selection of models in the situations of multicollinearity exploited.

This thesis is divided into two parts. In the first part, we show how problems of statistical inference and combinatorial optimization may be approached within a unified framework that employs tools from fields as diverse as machine learning, statistical physics and information theory, allowing us to i) design algorithms to solve the problems, ii) analyze the performance of these algorithms both empirically and analytically, and iii) to compare the results obtained with the optimal achievable ones. In the second part, we use this framework to study two specific problems, one of inference (compressed sensing) and the other of optimization (information hiding). In both cases, we review current approaches, identify their flaws, and propose new schemes to address these flaws, building on the use of message-passing algorithms, variational inference techniques, and spin glass models from statistical physics. / Esta tese está dividida em duas partes. Na primeira delas, mostramos como problemas de inferência estatística e de otimização combinatória podem ser abordados sob um framework unificado que usa ferramentas de áreas tão diversas quanto o aprendizado de máquina, a física estatística e a teoria de informação, permitindo que i) projetemos algoritmos para resolver os problemas, ii) analisemos a performance destes algoritmos tanto empiricamente como analiticamente, e iii) comparemos os resultados obtidos com os limites teóricos. Na segunda parte, este framework é usado no estudo de dois problemas específicos, um de inferência (compressed sensing) e outro de otimização (ocultação de dados). Em ambos os casos, revisamos abordagens recentes, identificamos suas falhas, e propomos novos esquemas que visam corrigir estas falhas, baseando-nos sobretudo em algoritmos de troca de mensagens, técnicas de inferência variacional, e modelos de vidro de spin da física estatística.

Gaussian processes can be used as priors on functions. The need for a flexible, principled, probabilistic model of functional relations is common in practice. Consequently, such an approach is demonstrably useful in a large variety of applications. Two challenges of Gaussian process modelling are often encountered. These are dealing with the adverse scaling with the number of data points and the lack of closed form posteriors when the likelihood is non-Gaussian. In this thesis, we study variational inference as a framework for meeting these challenges. An introductory chapter motivates the use of stochastic processes as priors, with a particular focus on Gaussian process modelling. A section on variational inference reviews the general definition of Kullback-Leibler divergence. The concept of prior conditional matching that is used throughout the thesis is contrasted to classical approaches to obtaining tractable variational approximating families. Various theoretical issues arising from the application of variational inference to the infinite dimensional Gaussian process setting are settled decisively. From this theory we are able to give a new argument for existing approaches to variational regression that settles debate about their applicability. This view on these methods justifies the principled extensions found in the rest of the work. The case of scalable Gaussian process classification is studied, both for its own merits and as a case study for non-Gaussian likelihoods in general. Using the resulting algorithms we find credible results on datasets of a scale and complexity that was not possible before our work. An extension to include Bayesian priors on model hyperparameters is studied alongside a new inference method that combines the benefits of variational sparsity and MCMC methods. The utility of such an approach is shown on a variety of example modelling tasks. We describe GPflow, a new Gaussian process software library that uses TensorFlow. Implementations of the variational algorithms discussed in the rest of the thesis are included as part of the software. We discuss the benefits of GPflow when compared to other similar software. Increased computational speed is demonstrated in relevant, timed, experimental comparisons.

En IRMf, les conclusions de paradigmes expérimentaux restent encore sujettes à caution dans la mesure où elles supposent une connaissance a priori du couplage neuro-vasculaire, c’est-à- dire de la fonction de réponse hémodynamique qui modélise le lien entre la stimulation et le signal mesuré. Afin de mieux appréhender les changements neuronaux et vasculaires induits par la réalisation d’une tâche cognitive en IRMf, il apparaît donc indispensable d’étudier de manière approfondie les caractéristiques de la réponse hémodynamique. Cette thèse apporte un nouvel éclairage sur cette étude, en s’appuyant sur une méthode originale d’analyse intra-sujet des données d’IRMf : la Détection-Estimation Conjointe (« Joint Detection-Estimation » en anglais, ou JDE). L’approche JDE modélise de façon non paramétrique et multivariée la réponse hémodynamique, tout en détectant conjointement les aires cérébrales activées en réponse aux stimulations d’un paradigme expérimental. La première contribution de cette thèse a été centrée sur l’analyse approfondie de la variabilité hémodynamique, tant inter-individuelle qu’inter-régionale, au niveau d’un groupe de jeunes adultes sains. Ce travail a permis de valider la méthode JDE au niveau d’une population et de mettre en évidence la variabilité hémodynamique importante apparaissant dans certaines régions cérébrales : lobes pariétal, temporal, occipital, cortex moteur. Cette variabilité est d’autant plus importante que la région est impliquée dans des processus cognitifs plus complexes.Un deuxième axe de recherche a consisté à se focaliser sur l’étude de l’organisation hémodynamique d’une aire cérébrale particulièrement importante chez les êtres humains, la région du langage. Cette fonction étant liée à la capacité d’apprentissage de la lecture, deux groupes d’enfants sains, âgés respectivement de 6 et 9 ans, en cours d’apprentissage ou de consolidation de la lecture, ont été choisis pour mener cette étude. Deux apports méthodologiques importants ont été proposés. Tout d’abord, une extension multi-sessions de l’approche JDE (jusqu’alors limitée au traitement de données mono-session en IRMf) a été mise au point afin d’améliorer la robustesse et la reproductibilité des résultats. Cette extension a permis de mettre en évidence, au sein de la population d’enfants, l’évolution de la réponse hémodynamique avec l’âge, au sein de la région du sillon temporal supérieur. Ensuite, un nouveau cadre a été développé pour contourner l’une des limitations de l’approche JDE « standard », à savoir la parcellisation a priori des données en régions fonctionnellement homogènes. Cette parcellisation est déterminante pour la suite de l’analyse et a un impact sur les résultats hémodynamiques. Afin de s’affranchir d’un tel choix, l’alternative mise au point combine les résultats issus de différentes parcellisations aléatoires des données en utilisant des techniques de «consensus clustering». Enfin, une deuxième extension de l’approche JDE a été mise en place pour estimer la forme de la réponse hémodynamique au niveau d’un groupe de sujets. Ce modèle a pour l’instant été validé sur simulations, et nous prévoyons de l’appliquer sur les données d’enfant pour améliorer l’étude des caractéristiques temporelles de la réponse BOLD dans les réseaux du langage.Ce travail de thèse propose ainsi d’une part des contributions méthodologiques nouvelles pour caractériser la réponse hémodynamique en IRMf, et d’autre part une validation et une application des approches développées sous un éclairage neuroscientifique. / In fMRI, the conclusions of experimental paradigms remain unreliable as far as they supposesome a priori knowledge on the neuro-vascular coupling which is characterized by thehemodynamic response function modeling the link between the stimulus input and the fMRIsignal as output. To improve our understanding of the neuronal and vascular changes inducedby the realization of a cognitive task given in fMRI, it seems thus critical to study thecharacteristics of the hemodynamic response in depth.This thesis gives a new perspective on this topic, supported by an original method for intra-subjectanalysis of fMRI data : the Joint Detection-Estimation (or JDE). The JDE approachmodels the hemodynamic response in a not parametric and multivariate manner, while itjointly detects the cerebral areas which are activated in response to stimulations deliveredalong an experimental paradigm.The first contribution of this thesis is centered on the thorough analysis of the interindividualand inter-regiona hemodynamic variability from a population of young healthyadults. This work has allowed to validate the JDE method at the group level and to highlightthe striking hemodynamic variability in some cerebral regions : parietal, temporal, occipitallobes, motor cortex. This variability is much more important as the region is involved in morecomplex cognitive processes.The second research axis has consisted in focusing on the study of the hemodynamic orga-nizationof a particularly important cerebral area in Humans, the language system. Becausethis function embeds the reading learning ability, groups of healthy children of 6 and 9 yearsold respectively, who were in the process of learning or of strenghting reading, were chosen forthis study. Two important methodological contributions have been proposed. First, a multi-sessionsextension of the JDE approach (until now limited to the processing of mono-sessiondata in fMRI) was worked out in order to improve the robustness and the reproducibility ofthe results. Then, a new framework was developed to overcome the main shortcoming of theJDE approach. The latter indeed relies on a prior parcellation of the data in functionally ho-mogeneousregions, the choice of which is critical for the subsequent inference and impacts thehemodynamic results. In order to avoid this a priori choice, the finalized alternative combinesthe results from various random data fragmentations by using “consensus clustering”.Finally, a second extension of the JDE approach was developed in order to robustly estimatethe shape of the hemodynamic response at the group level. So far, this model was validatedon simulations, and we plan to apply it on children data to improve the study of the BOLDresponse temporal characteristics in the language areas. Thus, this PhD work proposes onone hand new methodological contributions to characterize the hemodynamic response infMRI, and on the other hand a validation and a neuroscientific application of the proposedapproaches.

Este trabalho apresenta três principais contribuições: i. a proposta de uma nova lógica de descrição probabilística; ii. um novo algoritmo de inferência em primeira ordem a ser utilizado em terminologias representadas nessa lógica; e iii. aplicações práticas em problemas reais. A lógica aqui proposta, crALC (credal ALC), adiciona inclusões probabilísticas na popular lógica ALC combinando as terminologias com condições de aciclicidade, de Markov, e adotando uma semântica baseada em interpretações. Como os métodos de inferência exata tradicionalmente apresentam problemas de escalabilidade devido à presença de quantificadores (restrições universal e existencial), apresentamos um algoritmo de loopy propagation em primeira-ordem que se comporta bem para terminologias com domínios não triviais. Uma série de testes foi feita com o algoritmo proposto em comparação com algoritmos tradicionais da literatura; os resultados apresentados mostram uma clara vantagem em relação aos outros algoritmos. São apresentadas ainda duas aplicações da lógica e do algoritmo para resolver problemas reais da área de robótica móvel. Embora os problemas tratados sejam relativamente simples, eles constituem a base de muitos outros problemas da área, sendo um passo importante na representação de conhecimento de agentes/robôs autônomos e no raciocínio sobre esse conhecimento. / This work presents two major contributions: i. a new probabilistic description logic; ii. a new algorithm for inference in terminologies expressed in this logic; iii. practical applications in real tasks. The proposed logic, referred to as crALC (credal ALC), adds probabilistic inclusions to the popular logic ALC, combining the usual acyclicity and Markov conditions, and adopting interpretation-based semantics. As exact inference does not seem scalable due to the presence of quantifiers (existential and universal), we present a first-order loopy propagation algorithm that behaves appropriately for non-trivial domain sizes. A series of tests were done comparing the performance of the proposed algorithm against traditional ones; the presented results are favorable to the first-order algorithm. Two applications in the field of mobile robotics are presented, using the new probabilistic logic and the inference algorithm. Though the problems can be considered simple, they constitute the basis for many other tasks in mobile robotics, being a important step in knowledge representation and in reasoning about it.

We present a statistical image-based shape + structure model for Bayesian visual hull reconstruction and 3D structure inference. The 3D shape of a class of objects is represented by sets of contours from silhouette views simultaneously observed from multiple calibrated cameras. Bayesian reconstructions of new shapes are then estimated using a prior density constructed with a mixture model and probabilistic principal components analysis. We show how the use of a class-specific prior in a visual hull reconstruction can reduce the effect of segmentation errors from the silhouette extraction process. The proposed method is applied to a data set of pedestrian images, and improvements in the approximate 3D models under various noise conditions are shown. We further augment the shape model to incorporate structural features of interest; unknown structural parameters for a novel set of contours are then inferred via the Bayesian reconstruction process. Model matching and parameter inference are done entirely in the image domain and require no explicit 3D construction. Our shape model enables accurate estimation of structure despite segmentation errors or missing views in the input silhouettes, and works even with only a single input view. Using a data set of thousands of pedestrian images generated from a synthetic model, we can accurately infer the 3D locations of 19 joints on the body based on observed silhouette contours from real images.