We propose the idea of deep probabilistic programming, a synthesis of advances for systems at the intersection of probabilistic modeling and deep learning. Such systems enable the development of new probabilistic models and inference algorithms that would otherwise be impossible: enabling unprecedented scales to billions of parameters, distributed and mixed precision environments, and AI accelerators; integration with neural architectures for modeling massive and high-dimensional datasets; and the use of computation graphs for automatic differentiation and arbitrary manipulation of probabilistic programs for flexible inference and model criticism. After describing deep probabilistic programming, we discuss applications in novel variational inference algorithms and deep probabilistic models. First, we introduce the variational Gaussian process (VGP), a Bayesian nonparametric variational family, which adapts its shape to match complex posterior distributions. The VGP generates approximate posterior samples by generating latent inputs and warping them through random non-linear mappings; the distribution over random mappings is learned during inference, enabling the transformed outputs to adapt to varying complexity of the true posterior. Second, we introduce hierarchical implicit models (HIMs). HIMs combine the idea of implicit densities with hierarchical Bayesian modeling, thereby defining models via simulators of data with rich hidden structure.

Developments in modern bio-imaging techniques have allowed the routine collection of a vast amount of data from various techniques. The challenges lie in how to build accurate and efficient models to draw conclusions from the data and facilitate scientific discoveries. Fortunately, recent advances in statistics, machine learning, and deep learning provide valuable tools. This thesis describes some of our efforts to build scalable Bayesian models for four bio-imaging applications: (1) Stochastic Optical Reconstruction Microscopy (STORM) Imaging, (2) particle tracking, (3) voltage smoothing, (4) detect color-labeled neurons in c elegans and assign identity to the detections.

Le domaine de la théorie de la decision a été très actif depuis von Neumann Morgenstern 1943. De nouveaux modèles de décision ont révolutionné la manière avec laquelle on peut analyser nos actions et nos décisions. Cependant, le paradoxe de Allais en 1953 a obligé les théoriciens à clarifier l’objectif de leurs modèles. Alors de nombreux auteurs ont mis en avant le but normatif du modèle utilité espérée (les choix tel que nous devrions les faire, potentiellement meilleurs) et ont délaissé l’objectif descriptif (les choix réels, potentiellement biaisés).Cette évolution a permis a la discipline de définir de claire et solides méthodes de validation empirique de son approche descriptive. Cependant à l’inverse, la théorie de la decision normatif peine toujours à déterminer une méthodologie objective et constructive afin de trancher ses débats internes au sujet de la rationalité des modèles de théorie de la décision. Fournir une telle méthodologie est l’objectif principal de cette thèse. / Decision Theory has been a very dynamic field since von Neumann and Morgenstern 1943. New decision models have opened new ways to think about our actions and every day decisions.Allais’ Paradox in 1953 forced decision theorists to be clearer about the intents their models and several authors claimed that expected utility solely has a normative intent (choices that we should make, potentially better) and not a descriptive one (choices as we make them, potentially flawed).It also allowed to define better methods of validation for a descriptive point of view. Best practices in descriptive decision theory have emerged and we have now clear-cut and vetted methods of justifying the use of a given model of decision theory for a descriptive aim.However for normative decision theory that intents to help us make better choices, we do not have a clear cut way to determine and "prove" that a given model is the right one. This thesis provides an empirical design that provides such a methodology.

Decision support systems have emerged over five decades ago to serve decision makers in uncertain conditions and usually rapidly changing and unstructured problems. Most decision support approaches, such as Bayesian decision theory and computing with words, compare and analyze the consequences of different decision alternatives. Bayesian decision methods use probabilities to handle uncertainty and have been widely used in different areas for estimating, predicting, and offering decision supports. On the other hand, computing with words (CW) and approximate reasoning apply fuzzy set theory to deal with imprecise measurements and inexact information and are most concerned with propositions stated in natural language. The concept of a Z-number [69] has been recently introduced to represent propositions and their reliability in natural language. This work proposes a methodology that integrates Z-numbers and Bayesian decision theory to provide decision support when precise measurements and exact values of parameters and probabilities are not available. The relationships and computing methods required for such integration are derived and mathematically proved. The proposed hybrid methodology benefits from both approaches and combines them to model the expert knowledge and its certainty (reliability) in natural language and apply such model to provide decision support. To the best of our knowledge, so far there has been no other decision support methodology capable of using the reliability of propositions in natural language. In order to demonstrate the proof of concept, the proposed methodology has been applied to a realistic case study on breast cancer diagnosis and a daily life example of choosing means of transportation.

No description available.

With the ever-increasing availability of large, high-dimensional datasets, there is a growingneed for econometric methods that can handle such data. The last decade has seen the development of many such methods in computer science, but their applications to economic models have been limited. In this thesis, I investigate whether modern tools in (exact and approximate) Bayesian inference can be useful in economics. In the three chapters, my coauthors and I develop and estimate a variety of models applied to problems in organizational economics, health, and labor. In chapter one, joint with Andrew Olenski, we estimate a mortality-based Bayesian model of nursing home quality accounting for selection. We then conduct three exercises. First, we examine the correlates of quality, and find that public report cards have near-zero correlation. Second, we show that higher quality nursing homes fared better during the pandemic: a one standard deviation increase in quality corresponds to 2.5% fewer Covid-19 cases. Finally, we show that a 10% increase in the Medicaid reimbursement rate raises quality, leading to a 1.85 percentage point increase in 90-day survival. Such a reform would be cost-effective under conservative estimates of the quality-adjusted statistical value of life. In chapter two, joint with Laura Battaglia and Stephen Hansen, we demonstrate the effectiveness of Hamiltonian Monte Carlo (HMC) in analyzing high-dimensional data in a computationally efficient and methodologically sound manner. We propose a new model, called Supervised Topic Model with Covariates, that shows how modeling this type of data carefully can have significant implications on conclusions compared to a simpler yet methodologically problematic two-step approach. By conducting a simulation study and revisiting the study of executive time use by Bandiera, Prat, Hansen, and Sadun (2020), we demonstrate these results. This approach can accommodate thousands of parameters and doesn’t require custom algorithms specific to each model, making it more accessible for applied researchers. In chapter three, I propose a new way to estimate a two-way fixed effects model such as Abowd, Kramarz, and Margolis (1999) (AKM) that relaxes the stringent assumptions concerning the matching process. Through simulations, I demonstrate that this model performs well and provide an application to matched employer-employee data from Brazil. The results indicate that disregarding selection may result in a significant bias in the estimates of location fixed effects, and thus, can contribute to explaining recent discoveries about the relevance of locations in US labor markets. The three chapters demonstrate the usefulness of modern Bayesian methods for estimating models that would be otherwise infeasible, while remaining accessible enough for applied researchers. The importance of carefully modeling the data of interest instead of relying on ad-hoc solutions is also highlighted, as it has been shown to significantly impact the conclusions drawn across a variety of problems.

Learning theory is a dynamic and rapidly evolving field that aims to provide mathematical foundations for designing and understanding the behavior of algorithms and procedures that can learn from data automatically. At the heart of this field lies the interplay between algorithm design and statistical complexity analysis, with sharp statistical complexity characterizations often requiring localization analysis. This dissertation aims to advance the fields of machine learning and decision making by contributing to two key directions: principled algorithm design and localized statistical complexity. Our research develops novel algorithmic techniques and analytical frameworks to build more effective and robust learning systems. Specifically, we focus on studying uniform convergence and localization in statistical learning theory, developing efficient algorithms using the optimism principle for contextual bandits, and creating Bayesian design principles for bandit and reinforcement learning problems.

Indiana University-Purdue University Indianapolis (IUPUI) / After the initial network is constructed using expert’s knowledge of the domain, Bayesian networks need to be updated as and when new data is observed. Literature mining is a very important source of this new data. In this work, we explore what kind of data needs to be extracted with the view to update Bayesian Networks, existing technologies which can be useful in achieving some of the goals and what research is required to accomplish the remaining requirements. This thesis specifically deals with utilizing causal associations and experimental results which can be obtained from literature mining. However, these associations and numerical results cannot be directly integrated with the Bayesian network. The source of the literature and the perceived quality of research needs to be factored into the process of integration, just like a human, reading the literature, would. This thesis presents a general methodology for updating a Bayesian Network with the mined data. This methodology consists of solutions to some of the issues surrounding the task of integrating the causal associations with the Bayesian Network and demonstrates the idea with a semiautomated software system.

Indiana University-Purdue University Indianapolis (IUPUI) / Clinical Decision Support is one of the only aspects of health information technology that has demonstrated decreased costs and increased quality in healthcare delivery, yet it is extremely expensive and time-consuming to create, maintain, and localize. Consequently, a majority of health care systems do not utilize it, and even when it is available it is frequently incorrect. Therefore it is important to look beyond traditional guideline-based decision support to more readily available resources in order to bring this technology into widespread use. This study proposes that the wisdom of physicians within a practice is a rich, untapped knowledge source that can be harnessed for this purpose. I hypothesize and demonstrate that this wisdom is reflected by order entry data well enough to partially reconstruct the knowledge behind treatment decisions. Automated reconstruction of such knowledge is used to produce dynamic, situation-specific treatment suggestions, in a similar vein to Amazon.com shopping recommendations. This approach is appealing because: it is local (so it reflects local standards); it fits into workflow more readily than the traditional local-wisdom approach (viz. the curbside consult); and, it is free (the data are already being captured). This work develops several new machine-learning algorithms and novel applications of existing algorithms, focusing on an approach called Bayesian network structure learning. I develop: an approach to produce dynamic, rank-ordered situation-specific treatment menus from treatment data; statistical machinery to evaluate their accuracy using retrospective simulation; a novel algorithm which is an order of magnitude faster than existing algorithms; a principled approach to choosing smaller, more optimal, domain-specific subsystems; and a new method to discover temporal relationships in the data. The result is a comprehensive approach for extracting knowledge from order-entry data to produce situation-specific treatment menus, which is applied to order-entry data at Wishard Hospital in Indianapolis. Retrospective simulations find that, in a large variety of clinical situations, a short menu will contain the clinicians' desired next actions. A prospective survey additionally finds that such menus aid physicians in writing order sets (in completeness and speed). This study demonstrates that clinical knowledge can be successfully extracted from treatment data for decision support.

Indiana University-Purdue University Indianapolis (IUPUI) / Bayesian networks, one of the most widely used techniques to understand or predict the future by making use of current or previous data, have gained credence over the last decade for their ability to simulate large gene expression datasets to track and predict the reasons for changes in biological systems. In this work, we present a dynamic Bayesian model with gene annotation scores such as the gene characterization index (GCI) and the GenCards inferred functionality score (GIFtS) to understand and assess the prediction performance of the model by incorporating prior knowledge. Time-course breast cancer data including expression data about the genes in the breast cell-lines when treated with doxorubicin is considered for this study. Bayes server software was used for the simulations in a dynamic Bayesian environment with 8 and 19 genes on 12 different data combinations for each category of gene set to predict and understand the future time- course expression profiles when annotation scores are incorporated into the model. The 8-gene set predicted the next time course with r>0.95, and the 19-gene set yielded a value of r>0.8 in 92% cases of the simulation experiments. These results showed that incorporating prior knowledge into the dynamic Bayesian model for simulating the time- course expression data can improve the prediction performance when sufficient apriori parameters are provided.