No description available.

No description available.

No description available.

No description available.

<p dir="ltr"><i>In this dissertation, we present research results on two independent projects, one on </i><i>analysis of variance of multiple causal networks and the other on feature-specific coefficients </i><i>of determination in tree ensembles.</i></p>

No description available.

<p dir="ltr">In recent years, probabilistic graphical models have emerged as a powerful framework for understanding complex dependencies in multivariate data, offering a structured approach to tackle uncertainty and model complexity. These models have revolutionized the way we interpret the interplay between variables in various domains, from genetics to social network analysis. Inspired by the potential of probabilistic graphical models to provide insightful data analysis while addressing the challenges of high-dimensionality and computational efficiency, this dissertation introduces two novel methodologies that leverage the strengths of graphical models in high-dimensional settings. By integrating advanced inference techniques and exploiting the structural advantages of graphical models, we demonstrate how these approaches can efficiently decode complex data patterns, offering significant improvements over traditional methods. This work not only contributes to the theoretical advancements in the field of statistical data analysis but also provides practical solutions to real-world problems characterized by large-scale, complex datasets.</p><p dir="ltr">Firstly, we introduce a novel Bayesian hybrid method for learning the structure of Gaus- sian Bayesian Networks (GBNs), addressing the critical challenge of order determination in constraint-based and score-based methodologies. By integrating a permutation matrix within the likelihood function, we propose a technique that remains invariant to data shuffling, thereby overcoming the limitations of traditional approaches. Utilizing Cholesky decompo- sition, we reparameterize the log-likelihood function to facilitate the identification of the parent-child relationship among nodes without relying on the faithfulness assumption. This method efficiently manages the permutation matrix to optimize for the sparsest Cholesky factor, leveraging the Bayesian Information Criterion (BIC) for model selection. Theoretical analysis and extensive simulations demonstrate the superiority of our method in terms of precision, recall, and F1-score across various network complexities and sample sizes. Specifically, our approach shows significant advantages in small-n-large-p scenarios, outperforming existing methods in detecting complex network structures with limited data. Real-world applications on datasets such as ECOLI70, ARTH150, MAGIC-IRRI, and MAGIC-NIAB further validate the effectiveness and robustness of our proposed method. Our findings contribute to the field of Bayesian network structure learning by providing a scalable, efficient, and reliable tool for modeling high-dimensional data structures.</p><p dir="ltr">Secondly, we introduce a Bayesian methodology tailored for Gaussian Graphical Models (GGMs) that bridges the gap between GBNs and GGMs. Utilizing the Cholesky decomposition, we establish a novel connection that leverages estimated GBN structures to accurately recover and estimate GGMs. This innovative approach benefits from a theoretical foundation provided by a theorem that connects sparse priors on Cholesky factors with the sparsity of the precision matrix, facilitating effective structure recovery in GGMs. To assess the efficacy of our proposed method, we conduct comprehensive simulations on AR2 and circle graph models, comparing its performance with renowned algorithms such as GLASSO, CLIME, and SPACE across various dimensions. Our evaluation, based on metrics like estimation ac- curacy and selection correctness, unequivocally demonstrates the superiority of our approach in accurately identifying the intrinsic graph structure. The empirical results underscore the robustness and scalability of our method, underscoring its potential as an indispensable tool for statistical data analysis, especially in the context of complex datasets.</p>

<p>Deep learning has been the driving force behind many successful data science achievements. However, the deep neural network (DNN) that forms the basis of deep learning is</p> <p>often over-parameterized, leading to training, prediction, and interpretation challenges. To</p> <p>address this issue, it is common practice to apply an appropriate penalty to each connection</p> <p>weight, limiting its magnitude. This approach is equivalent to imposing a prior distribution</p> <p>on each connection weight from a Bayesian perspective. This project offers a systematic investigation into the selection of the penalty function or prior distribution. Specifically, under</p> <p>the general theoretical framework of posterior consistency, we prove that consistent sparse</p> <p>deep learning can be achieved with a variety of penalty functions or prior distributions.</p> <p>Examples include amenable regularization penalties (such as MCP and SCAD), spike-and?slab priors (such as mixture Gaussian distribution and mixture Laplace distribution), and</p> <p>polynomial decayed priors (such as the student-t distribution). Our theory is supported by</p> <p>numerical results.</p> <p><br></p>

<p dir="ltr">As machine learning continues to grow and surprise us, its complexity grows as well. Indeed, many machine learning models have become black boxes. Yet, there is a prevailing need for practicality. This dissertation offers some practicality on generative modeling and synthetic data, a recently popular application of generative models. First, Lightweight Chained Universal Approximators (LiCUS) is proposed. Motivated by statistical sampling principles, LiCUS tackles a simplified generative task with its universal approximation property while having a minimal computational bottleneck. When compared to a generative adversarial network (GAN) and variational auto-encoder (VAE), LiCUS empirically yields synthetic data with greater utility for a classifier on the Modified National Institute of Standards and Technology (MNIST) dataset. Second, following on its potential for informative synthetic data, LiCUS undergoes an extensive synthetic data supplementation experiment. The experiment largely serves as an informative starting point for practical use of synthetic data via LiCUS. In addition, by proposing a gold standard of reserved data, the experimental results suggest that additional data collection may generally outperform models supplemented with synthetic data, at least when using LiCUS. Given that the experiment was conducted on two datasets, future research could involve further experimentation on a greater number and variety of datasets, such as images. Lastly, generative machine learning generally demands large datasets, which is not guaranteed in practice. To alleviate this demand, one could offer expert knowledge. This is demonstrated by applying an expert-informed Wasserstein GAN with gradient penalty (WGAN-GP) on network flow traffic from NASA's Operational Simulation for Small Satellites (NOS3). If one were to directly apply a WGAN-GP, it would fail to respect the physical limitations between satellite components and permissible communications amongst them. By arming a WGAN-GP with cyber-security software Argus, the informed WGAN-GP could produce permissible satellite network flows when given as little as 10,000 flows. In all, this dissertation illustrates how machine learning processes could be modified under a more practical lens and incorporate pre-existing statistical principles and expert knowledge. </p>

<p dir="ltr">Deep neural networks (DNNs) have demonstrated outstanding performance in numerous tasks such as image recognition and natural language processing. However, in dynamic systems modeling, the tasks of estimating and uncovering the potentially nonlinear structure of systems represented by ordinary differential equations (ODEs) pose a significant challenge. In this dissertation, we employ DNNs to enable precise and efficient parameter estimation of dynamic systems. In addition, we introduce a highly flexible neural ODE model to capture both nonlinear and sparse dependent relations among multiple functional processes. Nonetheless, DNNs are susceptible to overfitting and often struggle to accurately assess predictive uncertainty despite their widespread success across various AI domains. The challenge of defining meaningful priors for DNN weights and characterizing predictive uncertainty persists. In this dissertation, we present a novel neural adaptive empirical Bayes framework with a new class of prior distributions to address weight uncertainty.</p><p dir="ltr">In the first part, we propose a precise and efficient approach utilizing DNNs for estimation and inference of ODEs given noisy data. The DNNs are employed directly as a nonparametric proxy for the true solution of the ODEs, eliminating the need for numerical integration and resulting in significant computational time savings. We develop a gradient descent algorithm to estimate both the DNNs solution and the parameters of the ODEs by optimizing a fidelity-penalized likelihood loss function. This ensures that the derivatives of the DNNs estimator conform to the system of ODEs. Our method is particularly effective in scenarios where only a set of variables transformed from the system components by a given function are observed. We establish the convergence rate of the DNNs estimator and demonstrate that the derivatives of the DNNs solution asymptotically satisfy the ODEs determined by the inferred parameters. Simulations and real data analysis of COVID-19 daily cases are conducted to show the superior performance of our method in terms of accuracy of parameter estimates and system recovery, and computational speed.</p><p dir="ltr">In the second part, we present a novel sparse neural ODE model to characterize flexible relations among multiple functional processes. This model represents the latent states of the functions using a set of ODEs and models the dynamic changes of these states utilizing a DNN with a specially designed architecture and sparsity-inducing regularization. Our new model is able to capture both nonlinear and sparse dependent relations among multivariate functions. We develop an efficient optimization algorithm to estimate the unknown weights for the DNN under the sparsity constraint. Furthermore, we establish both algorithmic convergence and selection consistency, providing theoretical guarantees for the proposed method. We illustrate the efficacy of the method through simulation studies and a gene regulatory network example.</p><p dir="ltr">In the third part, we introduce a class of implicit generative priors to facilitate Bayesian modeling and inference. These priors are derived through a nonlinear transformation of a known low-dimensional distribution, allowing us to handle complex data distributions and capture the underlying manifold structure effectively. Our framework combines variational inference with a gradient ascent algorithm, which serves to select the hyperparameters and approximate the posterior distribution. Theoretical justification is established through both the posterior and classification consistency. We demonstrate the practical applications of our framework through extensive simulation examples and real-world datasets. Our experimental results highlight the superiority of our proposed framework over existing methods, such as sparse variational Bayesian and generative models, in terms of prediction accuracy and uncertainty quantification.</p>