Ces travaux de recherche ont pour but de concevoir des ordinateurs baséssur une organisation du calcul mieux adaptée au raisonnement probabiliste.Notre intérêt s’est porté sur le traitement des données incertaines et lescalculs à réaliser sur celles-ci. Pour cela, nous proposons des architectures demachines se soustrayant au modèle Von Neumann, supprimant notammentl’utilisation de l’arithmétique en virgule fixe ou flottante. Les applicationscomme le traitement capteurs ou la robotique en générale sont des exemplesd’utilisation des architectures proposées.Plus spécifiquement, ces travaux décrivent deux types de machines probabilistes, radicalement différentes dans leur conception, dédiées aux problèmesd’inférences bayésiennes et utilisant le calcul stochastique. La première traiteles problèmes d’inférence de faibles dimensions et utilise le calcul stochas-tique pour réaliser les opérations nécessaires au calcul de l’inférence. Cettemachine est basée sur le concept de bus probabiliste et possède un très fortparallélisme. La deuxième machine permet de traiter les problèmes d’infé-rence en grandes dimensions. Elle implémente une méthode MCMC sous laforme d’un algorithme de Gibbs au niveau binaire. Dans ce cas, le calculstochastique est utilisé pour réaliser l’échantillonnage, bit à bit, du modèle.Une importante caractéristique de cette machine est de contourner les pro-blèmes de convergence généralement attribués au calcul stochastique. Nousprésentons en fin de manuscrit une extension de ce second type de machine :une machine générique et programmable permettant de trouver une solutionapprochée à n’importe quel problème d’inférence. / The aim of this research is to design computers best suited to do probabilistic reasoning. The focus of the research is on the processing of uncertain data and on the computation of probabilistic distribution. For this, new machine architectures are presented. The concept they are designed on is different to the one proposed by Von Neumann, without any fixed or floating point arithmetic. These architectures could replace the current processors in sensor processing and robotic fields.In this thesis, two types of probabilistic machines are presented. Their designs are radically different, but both are dedicated to Bayesian inferences and use stochastic computing. The first deals with small-dimension inference problems and uses stochastic computing to perform the necessary operations to calculate the inference. This machine is based on the concept of probabilistic bus and has a strong parallelism.The second machine can deal with intractable inference problems. It implements a particular MCMC method: the Gibbs algorithm at the binary level. In this case, stochastic computing is used for sampling the distribution of interest. An important feature of this machine is the ability to circumvent the convergence problems generally attributed to stochastic computing. Finally, an extension of this second type of machine is presented. It consists of a generic and programmable machine designed to approximate solution to any inference problem.

<p dir="ltr">The first chapter considers the l_α regularized linear regression, also termed Bridge regression. For α ∈ (0, 1), Bridge regression enjoys several statistical properties of interest such</p><p dir="ltr">as sparsity and near-unbiasedness of the estimates (Fan & Li, 2001). However, the main difficulty lies in the non-convex nature of the penalty for these values of α, which makes an</p><p dir="ltr">optimization procedure challenging and usually it is only possible to find a local optimum. To address this issue, Polson et al. (2013) took a sampling based fully Bayesian approach to this problem, using the correspondence between the Bridge penalty and a power exponential prior on the regression coefficients. However, their sampling procedure relies on Markov chain Monte Carlo (MCMC) techniques, which are inherently sequential and not scalable to large problem dimensions. Cross validation approaches are similarly computation-intensive. To this end, our contribution is a novel non-iterative method to fit a Bridge regression model. The main contribution lies in an explicit formula for Stein’s unbiased risk estimate for the out of sample prediction risk of Bridge regression, which can then be optimized to select the desired tuning parameters, allowing us to completely bypass MCMC as well as computation-intensive cross validation approaches. Our procedure yields results in a fraction of computational times compared to iterative schemes, without any appreciable loss in statistical performance.</p><p><br></p><p dir="ltr">Next, we build upon the classical and influential works of Neal (1996), who proved that the infinite width scaling limit of a Bayesian neural network with one hidden layer is a Gaussian process, when the network weights have bounded prior variance. Neal’s result has been extended to networks with multiple hidden layers and to convolutional neural networks, also with Gaussian process scaling limits. The tractable properties of Gaussian processes then allow straightforward posterior inference and uncertainty quantification, considerably simplifying the study of the limit process compared to a network of finite width. Neural network weights with unbounded variance, however, pose unique challenges. In this case, the classical central limit theorem breaks down and it is well known that the scaling limit is an α-stable process under suitable conditions. However, current literature is primarily limited to forward simulations under these processes and the problem of posterior inference under such a scaling limit remains largely unaddressed, unlike in the Gaussian process case. To this end, our contribution is an interpretable and computationally efficient procedure for posterior inference, using a conditionally Gaussian representation, that then allows full use of the Gaussian process machinery for tractable posterior inference and uncertainty quantification in the non-Gaussian regime.</p><p><br></p><p dir="ltr">Finally, we extend on the previous chapter, by considering a natural extension to deep neural networks through kernel processes. Kernel processes (Aitchison et al., 2021) generalize to deeper networks the notion proved by Neal (1996) by describing the non-linear transformation in each layer as a covariance matrix (kernel) of a Gaussian process. In this way, each succesive layer transforms the covariance matrix in the previous layer by a covariance function. However, the covariance obtained by this process loses any possibility of representation learning since the covariance matrix is deterministic. To address this, Aitchison et al. (2021) proposed deep kernel processes using Wishart and inverse Wishart matrices for each layer in deep neural networks. Nevertheless, the approach they propose requires using a process that does not emerge from the limit of a classic neural network structure. We introduce α-stable kernel processes (α-KP) for learning posterior stochastic covariances in each layer. Our results show that our method is much better than the approach proposed by Aitchison et al. (2021) in both simulated data and the benchmark Boston dataset.</p>

In this thesis we examine the question whether it is possible to model approach and avoidance behaviour with probabilistic machine learning. The results from this project will primarily aid in our collective understanding of human existence. Secondly, it will extend the knowledge with regards to probabilistic machine learning in the Neuroscience domain. We aid this through building a Variational Recurrent Neural Network (VRNN) that is trained on Electroencephalography (EEG)-data from participants that is subjected to odours with varying pleasantness. The pleasantness of the odours is used to divide the participants into two classes based on their self reported experience. This data is used to train the VRNN. The performance of the VRNN is evaluated by how well we are able to reconstruct the original data from a low dimensional latent representation. In this task the model performs on a similar level as related works. We further investigate how changes in the latent space effects reconstructed data. Despite being disentangled, the latent variables are hard to interpret. Furthermore we try to classify and cluster the latent space as either approach or avoidance behaviour with a Support Vector Machine and Uniform Manifold Approximation. The classification results are only slightly better than random, indicating that the learned latent space is not suitable for the task This is most likely due to the patterns that make up approach and avoidance behaviour is seen as noise by the VRNN. This leads to the patterns not being accurately modelled. This is shown by the evidence that frontal α -asymmetry that exists in the data is not reconstructed by the model. The conclusion is therefore that a VRNN is less suitable for modelling underlying behaviour from raw EEG data due to the low signal to noise ratio. We instead suggests to focus on specific frequency ranges in specific regions when applying machine learning in this domain. / Den här uppsatsen behandlar frågan huruvida det är möjligt att modellera närmande och frånstötande beteendemönster med hjälp av maskininlärning. Resultaten från detta projekt ämnar huvudsakligen att främja vidare förståelse av den mänskliga existensen. Vidare ämnar den även att utvidga förståelsen av hur probabilistisk maskininlärning kan användas för att utforska dylika hänseenden. Vi genomför detta genom att bygga en Variational Recurrent Neural Network-modell (VRNN) som tränas på data från experiment där personer utsätts för olika lukter samtidigt som deras Elektroencefalografi (EEG) spelas in. Deltagarna delas in i två klasser beroende på deras självrapporterade upplevelse av luktens njutbarhet. Maskininlärningsmodellen utvärderas genom att vi analyserar hur väl den lyckas rekonstruera datan. Detta lyckas den väl med. Vidare så undersöker vi hur förändringar i modellens latenta rum påverkar rekonstrueringen av datan. Resultaten från det experimentet är ej tydliga. Vidare så försöker vi klassificera och klustra det latenta rummet med avseende på närmande och frånstötande beteende med hjälp av en Support Vector Machine och Uniform Manifold Approximation. Resultaten från dessa experiment är att vi inte lyckas klassificera eller klustra det latenta rummet med avseende på närmande och frånstötande beteende bättre än slumpen. Vi argumenterar för att detta beror på att de underliggande mönster som skapar dessa beteenden ses som brus av VRNN-modellen och därmed inte modelleras. Detta visas genom att frontal α-asymmetri som existerar i datan ej rekonstrueras av modellen. Slutsaten blir därmed att en VRNN är mindre passande att använda vid modellering av underliggande beteenden av obehandlad EEG data. Detta på grund av det låga signal till brus-förhållandet i EEG-datan. Vi föreslår att istället fokusera på specifika frekvensområden i specifika hjärnregioner när maskininlärning appliceras på EEG.