This work proposes Hs-RuleFit, a learning method for regression and classification, which combines rule ensemble learning based on the RuleFit algorithm with Bayesian regularization through the horseshoe prior. To this end theoretical properties and potential problems of this combination are studied. A second step is the implementation, which utilizes recent sampling schemes to make the Hs-RuleFit computationally feasible. Additionally, changes to the RuleFit algorithm are proposed such as Decision Rule post-processing and the usage of Decision rules generated via Random Forest. Hs-RuleFit addresses the problem of finding highly accurate and yet interpretable models. The method shows to be capable of finding compact sets of informative decision rules that give a good insight in the data. Through the careful choice of prior distributions the horse-shoe prior shows to be superior to the Lasso in this context. In an empirical evaluation on 16 real data sets Hs-RuleFit shows excellent performance in regression and outperforms the popular methods Random Forest, BART and RuleFit in terms of prediction error. The interpretability is demonstrated on selected data sets. This makes the Hs-RuleFit a good choice for science domains in which interpretability is desired. Problems are found in classification, regarding the usage of the horseshoe prior and rule ensemble learning in general. A simulation study is performed to isolate the problems and potential solutions are discussed. Arguments are presented, that the horseshoe prior could be a good choice in other machine learning areas, such as artificial neural networks and support vector machines.

Gravitational lensing, a phenomenon in astronomy, occurs when the gravitational field of a massive object, such as a galaxy or a black hole, bends the path of light from a distant object behind it. This bending results in a distortion or magnification of the distant object's image, often seen as arcs or rings surrounding the foreground object. The Starlet wavelet transform offers a robust approach to representing galaxy images sparsely. This technique breaks down an image into wavelet coefficients at various scales and orientations, effectively capturing both large-scale structures and fine details. The Starlet wavelet transform offers a robust approach to representing galaxy images sparsely. This technique breaks down an image into wavelet coefficients at various scales and orientations, effectively capturing both large-scale structures and fine details. The horseshoe prior has emerged as a highly effective Bayesian technique for promoting sparsity and regularization in statistical modeling. It aggressively shrinks negligible values while preserving important features, making it particularly useful in situations where the reconstruction of an original image from limited noisy observations is inherently challenging. The main objective of this thesis is to apply sparse regularization techniques, particularly the horseshoe prior, to reconstruct the background source galaxy from gravitationally lensed images. By demonstrating the effectiveness of the horseshoe prior in this context, this thesis tackles the challenging inverse problem of reconstructing lensed galaxy images. Our proposed methodology involves applying the horseshoe prior to the wavelet coefficients of lensed galaxy images. By exploiting the sparsity of the wavelet representation and the noise-suppressing behavior of the horseshoe prior, we achieve well-regularized reconstructions that reduce noise and artifacts while preserving structural details. Experiments conducted on simulated lensed galaxy images demonstrate lower mean squared error and higher structural similarity with the horseshoe prior compared to alternative methods, validating its efficacy as an efficient sparse modeling technique. / Les lentilles gravitationnelles se produisent lorsque le champ gravitationnel d'un objet massif dévie la trajectoire de la lumière provenant d'un objet lointain, entraînant une distorsion ou une amplification de l'image de l'objet lointain. La transformation Starlet fournit une méthode robuste pour obtenir une représentation éparse des images de galaxies, capturant efficacement leurs caractéristiques essentielles avec un minimum de données. Cette représentation réduit les besoins de stockage et de calcul, et facilite des tâches telles que le débruitage, la compression et l'extraction de caractéristiques. La distribution a priori de fer à cheval est une technique bayésienne efficace pour promouvoir la sparsité et la régularisation dans la modélisation statistique. Elle réduit de manière agressive les valeurs négligeables tout en préservant les caractéristiques importantes, ce qui la rend particulièrement utile dans les situations où la reconstruction d'une image originale à partir d'observations bruitées est difficile. Étant donné la nature mal posée de la reconstruction des images de galaxies à partir de données bruitées, l'utilisation de la distribution a priori devient cruciale pour résoudre les ambiguïtés. Les techniques utilisant une distribution a priori favorisant la sparsité ont été efficaces pour relever des défis similaires dans divers domaines. L'objectif principal de cette thèse est d'appliquer des techniques de régularisation favorisant la sparsité, en particulier la distribution a priori de fer à cheval, pour reconstruire les galaxies d'arrière-plan à partir d'images de lentilles gravitationnelles. Notre méthodologie proposée consiste à appliquer la distribution a priori de fer à cheval aux coefficients d'ondelettes des images de galaxies lentillées. En exploitant la sparsité de la représentation en ondelettes et le comportement de suppression du bruit de la distribution a priori de fer à cheval, nous obtenons des reconstructions bien régularisées qui réduisent le bruit et les artefacts tout en préservant les détails structurels. Des expériences menées sur des images simulées de galaxies lentillées montrent une erreur quadratique moyenne inférieure et une similarité structurelle plus élevée avec la distribution a priori de fer à cheval par rapport à d'autres méthodes, validant son efficacité.

<p>Graphs are a natural choice for understanding the associations between variables, and assuming a probabilistic embedding for the graph structure leads to a variety of graphical models that enable us to understand these associations even further. In the realm of high-dimensional data, where the number of associations between interacting variables is far greater than the available number of data points, the goal is to infer a sparse graph. In this thesis, we make contributions in the domain of Bayesian graphical models, where our prior belief on the graph structure, encoded via uncertainty on the model parameters, enables the estimation of sparse graphs.</p> <p><br></p> <p>We begin with the Gaussian Graphical Model (GGM) in Chapter 2, one of the simplest and most famous graphical models, where the joint distribution of interacting variables is assumed to be Gaussian. In GGMs, the conditional independence among variables is encoded in the inverse of the covariance matrix, also known as the precision matrix. Under a Bayesian framework, we propose a novel prior--penalty dual called the `graphical horseshoe-like' prior and penalty, to estimate precision matrix. We also establish the posterior convergence of the precision matrix estimate and the frequentist consistency of the maximum a posteriori (MAP) estimator.</p> <p><br></p> <p>In Chapter 3, we develop a general framework based on local linear approximation for MAP estimation of the precision matrix in GGMs. This general framework holds true for any graphical prior, where the element-wise priors can be written as a Laplace scale mixture. As an application of the framework, we perform MAP estimation of the precision matrix under the graphical horseshoe penalty.</p> <p><br></p> <p>In Chapter 4, we focus on graphical models where the joint distribution of interacting variables cannot be assumed Gaussian. Motivated by the quantile graphical models, where the Gaussian likelihood assumption is relaxed, we draw inspiration from the domain of precision medicine, where personalized inference is crucial to tailor individual-specific treatment plans. With an aim to infer Directed Acyclic Graphs (DAGs), we propose a novel quantile DAG learning framework, where the DAGs depend on individual-specific covariates, making personalized inference possible. We demonstrate the potential of this framework in the regime of precision medicine by applying it to infer protein-protein interaction networks in Lung adenocarcinoma and Lung squamous cell carcinoma.</p> <p><br></p> <p>Finally, we conclude this thesis in Chapter 5, by developing a novel framework to compute the marginal likelihood in a GGM, addressing a longstanding open problem. Under this framework, we can compute the marginal likelihood for a broad class of priors on the precision matrix, where the element-wise priors on the diagonal entries can be written as gamma or scale mixtures of gamma random variables and those on the off-diagonal terms can be represented as normal or scale mixtures of normal. This result paves new roads for model selection using Bayes factors and tuning of prior hyper-parameters.</p>