In the modern era of high and infinite dimensional data, classical statistical methodology is often rendered inefficient and ineffective when confronted with such big data problems as arise in genomics, medical imaging, speech analysis, and many other areas of research. Many problems manifest when the practitioner is required to take into account the covariance structure of the data during his or her analysis, which takes on the form of either a high dimensional low rank matrix or a finite dimensional representation of an infinite dimensional operator acting on some underlying function space. Thus, novel methodology is required to estimate, analyze, and make inferences concerning such covariances. In this manuscript, we propose using tools from the concentration of measure literature–a theory that arose in the latter half of the 20th century from connections between geometry, probability, and functional analysis–to construct rigorous descriptive and inferential statistical methodology for covariance matrices and operators. A variety of concentration inequalities are considered, which allow for the construction of nonasymptotic dimension-free confidence sets for the unknown matrices and operators. Given such confidence sets a wide range of estimation and inferential procedures can be and are subsequently developed. For high dimensional data, we propose a method to search a concentration in- equality based confidence set using a binary search algorithm for the estimation of large sparse covariance matrices. Both sub-Gaussian and sub-exponential concentration inequalities are considered and applied to both simulated data and to a set of gene expression data from a study of small round blue-cell tumours. For infinite dimensional data, which is also referred to as functional data, we use a celebrated result, Talagrand’s concentration inequality, in the Banach space setting to construct confidence sets for covariance operators. From these confidence sets, three different inferential techniques emerge: the first is a k-sample test for equality of covariance operator; the second is a functional data classifier, which makes its decisions based on the covariance structure of the data; the third is a functional data clustering algorithm, which incorporates the concentration inequality based confidence sets into the framework of an expectation-maximization algorithm. These techniques are applied to simulated data and to speech samples from a set of spoken phoneme data. Lastly, we take a closer look at a key tool used in the construction of concentration based confidence sets: Rademacher symmetrization. The symmetrization inequality, which arises in the probability in Banach spaces literature, is shown to be connected with optimal transport theory and specifically the Wasserstein distance. This insight is used to improve the symmetrization inequality resulting in tighter concentration bounds to be used in the construction of nonasymptotic confidence sets. A variety of other applications are considered including tests for data symmetry and tightening inequalities in Banach spaces. An R package for inference on covariance operators is briefly discussed in an appendix chapter.

We investigate stability for certain geometric and functional inequalities and address the regularity of the free boundary for a problem arising in optimal transport theory. More specifically, stability estimates are obtained for the relative isoperimetric inequality inside convex cones and the Gaussian log-Sobolev inequality for a two parameter family of functions. Thereafter, away from a ``small" singular set, local C^{1,\alpha} regularity of the free boundary is achieved in the optimal partial transport problem. Furthermore, a technique is developed and implemented for estimating the Hausdorff dimension of the singular set. We conclude with a corresponding regularity theory on Riemannian manifolds. / text

Dans cette thèse, on a d’abord considéré une équation d'onde. On a premièrement montré que l’équation est bien-posée presque sûre par la méthode de décomposition de fréquence de Bourgain sous l’hypothèse de régularité que s > 2(p−3)/(p-1). Ensuite, nous avons réduit de cette exigence de régulation à (p-3)/(p−1) en appelant une estimation probabiliste a priori. Nous considérons également l’approximation des solutions obtenues ci-dessus par des solutions lisses et la stabilité de cette procédure d’approximation. Et nous avons conclu que l’équation est partout mal-posée dans le régime de super-critique. Nous avons considéré ensuite l’équation du faisceau quintique sur le tore 3D. Et nous avons montré que cette équation est presque sûr bien-posée globalement dans certain régimes de super-critique. Enfin, nous avons prouvé que la mesure de l’image de la mesure gaussienne sous l’application de flot de l’équation BBM généralisé satisfait une inégalité de type log-Sobolev avec une petit peu de perte de l’intégrabilité. / In this thesis, we consider a wave equation. We first showed that the equation is almost sure global well-posed via Bourgain’s high-low frequency decomposition under the regularity assumption s > 2(p−3)/(p−1). Then we lowered down this regularity requirement to be (p−3)/(p−1) by invoking a probabilistic a priori estimate. We also consider approximation of the above achieved solutions by smooth solutions and the stability of this approximating procedure. And we concluded that this equation is everywhere ill-posed in the super-critical regime. Next, we considered the quintic beam equation on 3D torus. And we showed that this equation is almost sure global well-posed in certain super-critical regime. Lastly, we proved that the image measure of the Gaussian measure under the generalized BBM flow map satisfies a log-Sobolev type inequality with a little bit loss of integrability.

<div>The rise of artificial intelligence (AI) hinges on the efficient training of modern deep neural networks (DNNs) for non-convex optimization and uncertainty quantification, which boils down to a non-convex Bayesian learning problem. A standard tool to handle the problem is Langevin Monte Carlo, which proposes to approximate the posterior distribution with theoretical guarantees. However, non-convex Bayesian learning in real big data applications can be arbitrarily slow and often fails to capture the uncertainty or informative modes given a limited time. As a result, advanced techniques are still required.</div><div><br></div><div>In this thesis, we start with the replica exchange Langevin Monte Carlo (also known as parallel tempering), which is a Markov jump process that proposes appropriate swaps between exploration and exploitation to achieve accelerations. However, the na\"ive extension of swaps to big data problems leads to a large bias, and the bias-corrected swaps are required. Such a mechanism leads to few effective swaps and insignificant accelerations. To alleviate this issue, we first propose a control variates method to reduce the variance of noisy energy estimators and show a potential to accelerate the exponential convergence. We also present the population-chain replica exchange and propose a generalized deterministic even-odd scheme to track the non-reversibility and obtain an optimal round trip rate. Further approximations are conducted based on stochastic gradient descents, which yield a user-friendly nature for large-scale uncertainty approximation tasks without much tuning costs. </div><div><br></div><div>In the second part of the thesis, we study scalable dynamic importance sampling algorithms based on stochastic approximation. Traditional dynamic importance sampling algorithms have achieved successes in bioinformatics and statistical physics, however, the lack of scalability has greatly limited their extensions to big data applications. To handle this scalability issue, we resolve the vanishing gradient problem and propose two dynamic importance sampling algorithms based on stochastic gradient Langevin dynamics. Theoretically, we establish the stability condition for the underlying ordinary differential equation (ODE) system and guarantee the asymptotic convergence of the latent variable to the desired fixed point. Interestingly, such a result still holds given non-convex energy landscapes. In addition, we also propose a pleasingly parallel version of such algorithms with interacting latent variables. We show that the interacting algorithm can be theoretically more efficient than the single-chain alternative with an equivalent computational budget.</div>