Scholars have recognized the benefits to science of Bayesian inference about the relative plausibility of competing hypotheses as opposed to, say, falsificationism in which one either rejects or fails to reject hypotheses in isolation. Yet inference about causal effects — at least as they are conceived in the potential outcomes framework (Neyman, 1923; Rubin, 1974; Holland, 1986) — has been tethered to falsificationism (Fisher, 1935; Neyman and Pearson, 1933) and difficult to integrate with Bayesian inference. One reason for this difficulty is that potential outcomes are fixed quantities that are not embedded in statistical models. Significance tests about causal hypotheses in either of the traditions traceable to Fisher (1935) or Neyman and Pearson (1933) conceive potential outcomes in this way; randomness in inferences about about causal effects stems entirely from a physical act of randomization, like flips of a coin or draws from an urn. Bayesian inferences, by contrast, typically depend on likelihood functions with model-based assumptions in which potential outcomes — to the extent that scholars invoke them — are conceived as outputs of a stochastic, data-generating model. In this dissertation, I develop Bayesian statistical inference for causal effects that incorporates the benefits of Bayesian scientific reasoning, but does not require probability models on potential outcomes that undermine the value of randomization as the “reasoned basis” for inference (Fisher, 1935, p. 14). In the first paper, I derive a randomization-based likelihood function in which Bayesian inference of causal effects is justified by the experimental design. I formally show that, under weak conditions on a prior distribution, as the number of experimental subjects increases indefinitely, the resulting sequence of posterior distributions converges in probability to the true causal effect. This result, typically known as the Bernstein-von Mises theorem, has been derived in the context of parametric models. Yet randomized experiments are especially credible precisely because they do not require such assumptions. Proving this result in the context of randomized experiments enables scholars to quantify how much they learn from experiments without sacrificing the design-based properties that make inferences from experiments especially credible in the first place. Having derived a randomization-based likelihood function in the first paper, the second paper turns to the calibration of a prior distribution for a target experiment based on past experimental results. In this paper, I show that usual methods for analyzing randomized experiments are equivalent to presuming that no prior knowledge exists, which inhibits knowledge accumulation from prior to future experiments. I therefore develop a methodology by which scholars can (1) turn results of past experiments into a prior distribution for a target experiment and (2) quantify the degree of learning in the target experiment after updating prior beliefs via a randomization-based likelihood function. I implement this methodology in an original audit experiment conducted in 2020 and show the amount of Bayesian learning that results relative to information from past experiments. Large Bayesian learning and statistical significance do not always coincide, and learning is greatest among theoretically important subgroups of legislators for which relatively less prior information exists. The accumulation of knowledge about these subgroups, specifically Black and Latino legislators, carries implications about the extent to which descriptive representation operates not only within, but also between minority groups. In the third paper, I turn away from randomized experiments toward observational studies, specifically the Difference-in-Differences (DID) design. I show that DID’s central assumption of parallel trends poses a neglected problem for causal inference: Counterfactual uncertainty, due to the inability to observe counterfactual outcomes, is hard to quantify since DID is based on parallel trends, not an as-if-randomized assumption. Hence, standard errors and ?-values are too small since they reflect only sampling uncertainty due to the inability to observe all units in a population. Recognizing this problem, scholars have recently attempted to develop inferential methods for DID under an as-if-randomized assumption. In this paper, I show that this approach is ill-suited for the most canonical DID designs and also requires conducting inference on an ill-defined estimand. I instead develop an empirical Bayes’ procedure that is able to accommodate both sampling and counterfactual uncertainty under the DIDs core identification assumption. The overall method is straightforward to implement and I apply it to a study on the effect of terrorist attacks on electoral outcomes.

The ultimate goal of personalized or precision medicine is to identify the best treatment for each patient. An N-of-1 trial is a multiple-period crossover trial performed within a single individual, which focuses on individual outcome instead of population or group mean responses. As in a conventional crossover trial, it is critical to understand carryover effects of the treatment in an N-of-1 trial, especially in situations where there are no washout periods between treatment periods and high volume of measurements are made during the study. Existing statistical methods for analyzing N-of-1 trials include nonparametric tests, mixed effect models and autoregressive models. These methods may fail to simultaneously handle measurements autocorrelation and adjust for potential carryover effects. Distributed lag model is a regression model that uses lagged predictors to model the lag structure of exposure effects. In the dissertation, we first introduce a novel Bayesian distributed lag model that facilitates the estimation of carryover effects for single N-of-1 trial, while accounting for temporal correlations using an autoregressive model. In the second part, we extend the single N-of-1 trial model to multiple N-of-1 trials scenarios. In the third part, we again focus on single N-of-1 trials. But instead of modeling comparison with one treatment and one placebo (or active control), multiple treatments and one placebo (or active control) is considered. In the first part, we propose a Bayesian distributed lag model with autocorrelated errors (BDLM-AR) that integrate prior knowledge on the shape of distributed lag coefficients and explicitly model the magnitude and duration of carryover effect. Theoretically, we show the connection between the proposed prior structure in BDLM-AR and frequentist regularization approaches. Simulation studies were conducted to compare the performance of our proposed BDLM-AR model with other methods and the proposed model is shown to have better performance in estimating total treatment effect, carryover effect and the whole treatment effect coefficient curve under most of the simulation scenarios. Data from two patients in the light therapy study was utilized to illustrate our method. In the second part, we extend the single N-of-1 trial model to multiple N-of-1 trials model and focus on estimating population level treatment effect and carryover effect. A Bayesian hierarchical distributed lag model (BHDLM-AR) is proposed to model the nested structure of multiple N-of-1 trials within the same study. The Bayesian hierarchical structure also improve estimates for individual level parameters by borrowing strength from the N-of-1 trials of others. We show through simulation studies that BHDLM-AR model has best average performance in terms of estimating both population level and individual level parameters. The light therapy study is revisited and we applied the proposed model to all patients’ data. In the third part, we extend BDLM-AR model to multiple treatments and one placebo (or active control) scenario. We designed prior precision matrix on each treatment. We demonstrated the application of the proposed method using a hypertension study, where multiple guideline recommended medications were involved in each single N-of-1 trial.

In this dissertation I study how different cognitive constraints affect individuals' belief formation process, and the consequences of these constraints on behavior. In the first chapter I present laboratory experiments designed to test whether subjects' inability to perform more rounds of iterated deletion of dominated strategies is due to cognitive limitations, or to higher order beliefs about the rationality of others. I propose three alternative explanations for why subjects might not be doing more iterations of dominance reasoning. First, they might have problems computing iterated best responses, even when doing so does not require higher order beliefs. Second, subjects might face limitations in their ability to generate higher order beliefs. Finally, subjects' behavior might not be limited by cognitive limitations, but rather justified by their beliefs about what others will play. I design two experiments in order to test these hypothesis. Findings from the first experiment suggest that most subjects' strategies (about 66%) are not the result of their inability to compute iterated best responses. I then run a second experiment, finding that about 70% of the subjects' behavior come from limitations in their ability to iterate best responses and generate higher order beliefs at the same time, while for the other 30% their strategies are a best response to higher order beliefs that others are not rational. In the second chapter I study whether a Sender in a Bayesian Persuasion setting (Kamenica and Gentzkow, 2011) can benefit from behavioral biases in the way Receivers update their beliefs, by choosing how to communicate information. I present three experiments in order to test this hypothesis, finding that Receivers tend to overestimate the probability of a state of the world after receiving signals that are more likely in that state. Because of this bias, Senders' gains from persuasion can be increased by ``muddling the water'' and making it hard for Receivers to find the correct posteriors. This contradicts the theoretical result that states that communicating using signal structures is equivalent to communicating which posteriors these structures induce. Through analysis of the data and robustness experiments, I am able to discard social preferences or low incentives as driving my results, leaving base-rate neglect as a more likely explanation. The final chapter studies whether sensory bottlenecks, as oppose to purely computational cognitive constraints, are important factors affecting subjects' inference in an experiment that mimics financial markets. We show that providing redundant visual and auditory cues about the liquidity of a stock significantly improves performance, corroborating previous findings in neuroscience of multi-sensory integration, which could have policy implications in economically relevant situation.

This dissertation consists of three essays on the design of regulatory systems intended to inform market participants about product quality. The central theme is how asymmetric information problems influence the incentives of customers, regulated firms, and certifiers, and the implications these distortions have for welfare and market design. The first chapter, Regulation by Information Provision, studies quality provision in New York City's elevator maintenance market. In this market, service providers maintain machines and are inspected periodically by city inspectors. I find evidence that monitoring frictions create moral hazard for service providers. In the absence of perfect monitoring, buildings rely on signals generated by the regulator to hold service providers accountable, cancelling contracts when bad news arrives and preserving them when good news arrives. Regulatory instruments, such as inspection frequency and fine levels, can therefore influence provider effort in two ways: (i) by directly changing the cost of effort (e.g. fines for poor peformance); (ii) by changing expected future revenue (through building cancellation decisions). Using a structural search model of the industry, I find that the second channel is the dominant one. In particular, I note that strengthening the information channel has two equilibrium effects: first, it increases provider effort; and second, it shifts share towards higher-quality matches since buildings can more quickly sever unproductive relationships. These findings have important policy implications, as they suggest that efficient information provision --- for example, targeting inspections to newly-formed relationships --- is a promising avenues for welfare improvement. The second chapter, Quality Disclosure Design, studies a similar regulatory scheme, but emphasizes the incentives of the certifier. In particular, I argue that restaurant inspectors in New York City are locally averse to giving restaurants poor grades: restaurants whose inspections are on the border of an A versus a B grade are disproportionately given an A. The impact of this bias is twofold: first, it degrades the quality of the information provided to the market, as there is substantial heterogeneity in food-poisoning risk even within A restaurants. Second, by making it easier to achieve passing grades, inspector bias reduces incentives for restaurants to invest in their health practices. After developing a model of the inspector-restaurant interaction, counterfactual work suggests that stricter grading along the A-B boundary could generate substantial improvements in food-poisoning rates. The policy implications of these findings depends on the source of inspector bias. I find some evidence that bias is bureaucratic in nature: when inspectors have inspection decisions overturned in an administrative trial, they are more likely to score leniently along the A-B boundary in their other inspections. However, it's not clear whether this behavior stems from administrative burden (a desire to avoid more trials) or a desire to avoid looking incompetent. Pilot programs that reduce the administrative burden of giving B grades are a promising avenue for future research. The last chapter, Real-Time Inference, also studies the incentives of certifiers, namely MLB umpires charged with classifying pitches as balls or strikes. Unlike in \textit{Quality Disclosure Design}, I find that umpire ball/strike decisions are remarkably bias-free. Previous literature on this topic has noted a tendency for umpires to --- for a fixed pitch location --- call more strikes in hitter's counts and more balls in pitcher's counts. I propose a simple rational explanation for this behavior: umpires are Bayesian. In hitter's counts, such as 3-0, pitchers tend to throw pitches right down the middle of the plate, whereas in pitcher's counts, they throw pitches outside the strike zone. For a borderline pitch, the umpire's prior will push it towards the strike zone in a 3-0 count and away from the strike-zone in an 0-2 count, producing the exact divergence in ball/strike calls noted in previous work. While implications for broader policy are not immediately obvious, I note several features of the environment that are conducive to umpires effectively approximating optimal inference, particularly the frequent, data-driven feedback that umpires receive on their performance.

Thesis (M.Sc. (Statistics)) -- University of Limpopo, 2020 / Malaria is a mosquito borne disease, a major cause of human morbidity and mortality in most of the developing countries in Africa. South Africa is one of the countries with high risk of malaria transmission, with many cases reported in Mpumalanga and Limpopo provinces. Bayesian and classical methods of estimation have been applied and compared on the effect of climatic factors (rainfall, temperature, normalised difference vegetation index, and elevation) on malaria incidence. Credible and confidence intervals from a negative binomial model estimated via Bayesian estimation-Markov chain Monte Carlo process and maximum likelihood, respectively, were utilised in the comparison process. Bayesian methods appeared to be better than the classical method in analysing malaria incidence in the Limpopo province of South Africa. The classical framework identified rainfall and temperature during the night to be the significant predictors of malaria incidence in Mopani, Vhembe and Waterberg districts of Limpopo province. However, the Bayesian method identified rainfall, normalised difference vegetation index, elevation, temperature during the day and temperature during the night to be the significant predictors of malaria incidence in Mopani, Sekhukhune, Vhembe and Waterberg districts of Limpopo province. Both methods also affirmed that Vhembe district is more susceptible to malaria incidence, followed by Mopani district. We recommend that the Department of Health and Malaria Control Programme of South Africa allocate more resources for malaria control, prevention and elimination to Vhembe and Mopani districts of Limpopo province. Future research may involve studies on the methods to select the best prior distributions. / National Research Foundation (NRF)

This dissertation studies problems in individual and collective decision making. Chapter 1 examines how information providers may compete to influence the actions of one or many decision makers. This chapter studies a Bayesian Persuasion game with multiple senders who have access to conditionally independent experiments (and possibly others). Senders have zero-sum preferences over what information is revealed. The main results characterize when any set of states can be pooled in equilibrium and, as a consequence, when the state is (fully) revealed in every equilibrium. The state must be fully revealed in every equilibrium if and only if sender utility functions satisfy a ‘global nonlinearity’ condition. In the binary-state case, the state is fully revealed in every equilibrium if and only if some sender has nontrivial preferences. Our main takeaway is that ‘most’ zero-sum sender preferences result in full revelation. We discuss a number of extensions and variations. Chapter 2 studies Liquid Democracy (LD), a voting system which combines aspects of direct democracy (DD) and representative democracy (RD) and is becoming more widely used for collective decision making. In LD, for every decision each voter is endowed with a vote and can cast it themselves or delegate it to another voter. We study information aggregation under LD in a common-interest jury voting game with heterogenously well-informed voters. There is an incentive for a voter i to delegate to someone better informed; but delegation has a cost: if i delegates her vote, she can no longer express her own private information by voting. Delegation trades off empowering better information and making use of more information. Under some conditions, efficiency requires the number of votes held by each nondelegator to optimally reflect how well informed they are. Under efficiency LD improves welfare over DD and RD, especially in medium-sized committees. However LD also admits inefficient equilibria characterized by a small number of voters holding a large share of votes. Such equilibria can do worse than DD and can fail to aggregate information asymptotically. We discuss the implications of our results for implementing LD. For many years, psychologists have discussed the possibility of choice overload: large choice sets can be detrimental to a chooser’s wellbeing. The existence of such a phenomenon would have profound impact on both the positive and normative study of economic decision making, yet recent meta studies have reported mixed evidence. In Chapter 3, we argue that existing tests of choice overload - as measured by an increased probability of choosing a default option - are likely to be significantly under powered because ceteris parabus we should expect the default alternative to be chosen less often in larger choice sets. We propose a more powerful test based on richer data and characterization theorems for the Random Utility Model. These new approaches come with significant econometric challenges, which we show how to address. We apply the resulting tests to an exploratory data set of choices over lotteries.

This dissertation studies statistical decision making and belief formation in face of uncertainty, that is, when agents' payoffs depend on an unknown distribution. Chapter 1 introduces and analyzes an equilibrium solution concept in which players sequentially sample to resolve strategic uncertainty over their opponents' distribution of actions. Bayesian players can sample from their opponents' distribution of actions at a cost and make optimal choices given their posterior beliefs. The solution concept makes predictions on the joint distribution of players' choices, beliefs, and decision times, and generates stochastic choice through the randomness inherent to sampling, without relying on indifference or choice mistakes. It rationalizes well-known deviations from Nash equilibrium such as the own-payoff effect and I show its novel predictions relating choices, beliefs, and decision times are supported by existing data. Chapter 2 presents experimental evidence establishing that the level of incentives affects both gameplay and mean beliefs.Holding fixed the actions of the other player, it is shown that, in the context of a novel class of dominance-solvable games --- diagonal games ---, higher incentives make subjects more likely to best-respond to their beliefs. Moreover, higher incentives result in more responsive beliefs but not necessarily less biased. Incentives affect effort --- as proxied by decision time --- and that it is effort, and not incentives directly, that accounts for the changes in belief formation. The results support models where, in addition to choice mistakes, players exhibit costly attention. Chapter 3 examines the class of diagonal games that are used in Chapter 2. Diagonal games constitute a new class of two-player dominance-solvable games which constitutes a useful benchmark in the study of cognitive limitations in strategic settings, both for exploring predictions of theoretical models and for experiments. This class of finite games allows for a disciplined way to vary two features of the strategic setting plausibly related to game complexity: the number of steps of iterated elimination of dominated actions required to reach the dominance solution and the number of actions. Furthermore, I derive testable implications of solution concepts such as level-k, endogenous depth of reasoning, sampling equilibrium, and quantal response equilibrium. Finally, Chapter 4 studies the robustness of pricing strategies when a firm is uncertain about the distribution of consumers' willingness-to-pay. When the firm has access to data to estimate this distribution, a simple strategy is to implement the mechanism that is optimal for the estimated distribution. We find that such empirically optimal mechanism delivers exponential, finite-sample profit and regret guarantees. Moreover, we provide a toolkit to evaluate the robustness properties of different mechanisms, showing how to consistently estimate and conduct valid inference on the profit generated by any one mechanism, which enables one to evaluate and compare their probabilistic revenue guarantees.

We propose the idea of deep probabilistic programming, a synthesis of advances for systems at the intersection of probabilistic modeling and deep learning. Such systems enable the development of new probabilistic models and inference algorithms that would otherwise be impossible: enabling unprecedented scales to billions of parameters, distributed and mixed precision environments, and AI accelerators; integration with neural architectures for modeling massive and high-dimensional datasets; and the use of computation graphs for automatic differentiation and arbitrary manipulation of probabilistic programs for flexible inference and model criticism. After describing deep probabilistic programming, we discuss applications in novel variational inference algorithms and deep probabilistic models. First, we introduce the variational Gaussian process (VGP), a Bayesian nonparametric variational family, which adapts its shape to match complex posterior distributions. The VGP generates approximate posterior samples by generating latent inputs and warping them through random non-linear mappings; the distribution over random mappings is learned during inference, enabling the transformed outputs to adapt to varying complexity of the true posterior. Second, we introduce hierarchical implicit models (HIMs). HIMs combine the idea of implicit densities with hierarchical Bayesian modeling, thereby defining models via simulators of data with rich hidden structure.

Developments in modern bio-imaging techniques have allowed the routine collection of a vast amount of data from various techniques. The challenges lie in how to build accurate and efficient models to draw conclusions from the data and facilitate scientific discoveries. Fortunately, recent advances in statistics, machine learning, and deep learning provide valuable tools. This thesis describes some of our efforts to build scalable Bayesian models for four bio-imaging applications: (1) Stochastic Optical Reconstruction Microscopy (STORM) Imaging, (2) particle tracking, (3) voltage smoothing, (4) detect color-labeled neurons in c elegans and assign identity to the detections.

No description available.