In 2023, breakthroughs achieved by large language models like ChatGPT have been transformative, revealing the hidden structures within natural language. This has enabled these models to reason and perform tasks with intelligence previously unattainable, a feat made possible by analyzing vast datasets. However, the domain of medical imaging—characterized by the high costs and intensive labor of data acquisition, along with the scarcity of data from pathological sources—presents unique challenges. Neuroimaging data, for instance, is marked by its high dimensionality, limited sample sizes, complex hierarchical and temporal structures, significant noise, and contextual variability. These obstacles are especially prevalent in methodologies like functional Magnetic Resonance Imaging (fMRI) and computer vision applications, where datasets are naturally sparse. Developing sophisticated methods to overcome these challenges is essential for maximizing the utility of imaging technologies and enhancing our understanding of neurological functions. Such advancements are critical for the creation of innovative diagnostic tools and therapeutic approaches for neurological and psychiatric conditions. The data from current set of non-invasive neuroimaging modalities is most often analyzed using classical statistical and machine learning methods. In this work we show that widely used machine learning methods for neural imaging data can be unified under a Bayesian perspective. We use this unifying view of probabilistic modeling techniques to further develop models and statistical inference methods to address the aforementioned challenges by leveraging substantial research developments in artificial intelligence i.e. deep learning, and probabilistic modeling methods over the last decade. In this work, we broaden the family of probabilistic models to encompass various prior structures,including discrete, hierarchical, and temporal elements. We derive efficient inference models using principled Bayesian inference and modern stochastic optimization and empirically demonstrate how the representational capacity of neural networks can be combined with principled probabilistic generative models to achieve state-of-the-art results on neuroimaging and computer vision datasets. The methods we develop are applicable to a diverse range of datasets beyond neuroimaging; for instance, we apply these probabilistic inference principles to improve movie and song recommendations, enhance object detection in computer vision models, and perform neural architecture search.

In recent years, markets have evolved due to the disruption of digital marketplaces, and the rise of concerns about fairness, accountability and privacy. These changes have introduced new challenges for market designers. In this dissertation, we study the design and optimization of different markets. For each market, we provide a theoretical framework to analyze current solutions. Furthermore, we propose alternative solutions and identify the trade-offs between efficiency and other goals. In the first part of this dissertation, we study markets where tickets for a shared experience are allocated through a lottery. A group of agents is successful if and only if its members receive enough tickets for everyone. We study the efficiency and fairness of existing lottery mechanisms and propose practical alternatives. If agents must identify the members of their group, a natural solution is the Group Lottery, which orders groups uniformly at random and processes them sequentially. We show that the Group Lottery is approximately fair and approximately efficient. If agents may request multiple tickets without identifying members of their group, the most common mechanism is the Individual Lottery, which orders agents uniformly at random and awards each their request until no tickets remain. This approach can yield arbitrarily unfair and inefficient outcomes. As an alternative, we propose the Weighted Individual Lottery, in which the processing order is biased against agents with large requests. This simple modification makes the Weighted Individual Lottery  approximately fair and approximately efficient, and similar to the Group Lottery when there are many more agents than tickets. The second part of the dissertation focuses on markets in which an organization is presented with a set of individuals and must choose which subset to accept. The organization makes a selection based on a priority ranking of individuals as well as other observable characteristics. We propose the outcome based selection rules, which are defined by a collection of feasible selections and a greedy processing algorithm. For these rules, we (i) provide an axiomatic characterization, (ii) show that it chooses the only selection that respects priorities, and (iii) identify several cases where is efficient (choose the feasible selection with the highest value). Finally, we connect these ideas with the Chilean Constitutional Assembly election, and show that the rule that was implemented in practice is an outcome based selection rule. In the third part of this work, we study digital marketplaces where an online platform maximizes its revenue by influencing consumer buying behavior through the disclosure of information. In this market, consumers need to engage in a costly search process to acquire additional information. We develop a new model that combines a Bayesian persuasion problem with an optimal sequential search framework inspired by Weitzman's 1979. We characterize the platform's optimal policy under the assumption that the platform must provide a certain level of disclosure to incentivize the consumer to investigate. The optimal policy uses a binary signal indicating whether the item is a good match for the consumer or not. Additionally, we provide a conjecture on the platform's optimal policy when the assumption is relaxed and there are only two items. The structure of the optimal policy depends on the consumer's prior beliefs about the items and how they compare with the value of the outside option. However, in all scenarios, the optimal signals are either binary or uninformative. This conjecture is supported by a numerical analysis performed on a novel formulation based on quadratic programming.

In this thesis, we present probabilistic approaches for two critical aspects of deep learning: unsupervised representation learning and uncertainty estimation. The first part of the thesis focuses on developing a probabilistic method for deep representation learning and an application of representation learning on multimodal text-video data. Unsupervised representation learning has been proven effective for learning useful representations of data using deep learning and enhancing the performance on downstream applications. However, current methods for representation learning lack a solid theoretical foundation despite their empirical success. To bridge this gap, we present a novel perspective for unsupervised representation learning: we argue that representation learning should maximize the effective nonlinear expressivity of a deep neural network on the data so that the downstream predictors can take full advantage of its nonlinear representation power. To this end, we propose our method of neural activation coding (NAC) that maximizes the mutual information between activation patterns of the encoder and the data over a noisy communication channel. We show that learning for a noise-robust activation code maximizes the number of distinct linear regions of ReLU encoders, hence maximizing its nonlinear expressivity. Experiment results demonstrate that NAC enhances downstream performance on linear classification and nearest neighbor retrieval on natural image datasets, and furthermore significantly improve the training of deep generative models. Next, we study an application of representation learning for multimodal text-video retrieval. We reveal that when using a pretrained representation model, many test instances are either over- or under-represented during text-video retrieval, hurting the retrieval performance. To address the problem, we propose normalized contrastive learning (NCL) that utilizes the Sinkhorn-Knopp algorithm to normalize the retrieval probabilities of text and video instances, thereby significantly enhancing the text-video retrieval performance. The second part of the thesis addresses the critical challenge of quantifying the predictive uncertainty of deep learning models, which is crucial for high-stakes applications of ML including medical diagnosis, autonomous driving, and financial forecasting. However, uncertainty estimation for deep learning remains an open challenge and current Bayesian approximations often output unreliable uncertainty estimates. We propose a density-based uncertainty criterion that posits that a model’s predictive uncertainty should be grounded in the density of the model’s training data so that the predictive uncertainty is high for inputs that are unlikely under the training data distribution. To this end, we introduce density uncertainty layers as a general building block for density-aware deep architectures. These layers embed the density-based uncertainty criterion directly into the model architecture and can be used as a drop-in replacement for existing neural network layers to produce reliable uncertainty estimates for deep learning models. On uncertainty estimation benchmarks, we show that the proposed method delivers more reliable uncertainty estimates and robust out-of-distribution detection performance.

One of the major mysteries in science is the remarkable success of machine learning. While its empirical achievements have captivated the field, our theoretical comprehension lags significantly behind. This thesis seeks for advancing our theoretical understanding of learning and intelligence from a computational perspective. By studying fundamental learning, optimization and decision making tasks, we aspire to shed lights on the impact of computation for artificial intelligence. The first part of this thesis concerns the space resource needed for learning. By studying the fundamental role of memory for continual learning, online decision making and convex optimization, we find both continual learning and efficient convex optimization require a lot of memory; while for decision making, exponential savings are possible. More concretely, (1) We prove there is no memory deduction in continual learning, unless the continual learner takes multiple passes over the sequence of environments; (2) We prove in order to optimize a convex function in the most iteration efficient way, an algorithm must use a quadratic amount of space; (3) We show polylogarithmic space is sufficient for making near optimal decisions in an oblivious adversary environment; in a sharp contrast, a quadratic saving is both sufficient and necessary to achieve vanishing regret in an adaptive adversarial environment. The second part of this thesis uses learning as a tool, and resolves a series of long-standing open questions in algorithmic game theory. By giving an exponential faster no-swap-regret learning algorithm, we obtain algorithms that achieve near-optimal computation/communication/iteration complexity for computing an approximate correlated equilibrium in a normal-form game, and we give the first polynomial-time algorithm for computing approximate normal-form correlated equilibria in imperfect information games (including Bayesian and extensive-form games).

Approved for public release; distribution in unlimited. / The principles of Sun Tzu's Art of War have been widely used by business executives and military officers with much success in the realm of competition and conflict. However, when conflict situations arise in a highly stressful environment coupled with the pressure of time, decision makers may not be able to consider all the key concepts when forming their decisions or strategies. Therefore, a structured reasoning approach may be used to apply Sun Tzu's principles correctly and fully. Sun Tzu's principles are believed to be able to be modeled mathematically; hence, a Bayesian Network model (a form of mathematical tool using probability theory) is used to capture Sun Tzu's principles and provide the structured reasoning approach. Scholars have identified incompleteness in Sun Tzu's appreciation of information in war and his application of secret agents. This incompleteness resulted in circular reasoning when both sides of the conflict apply his principles. This circular reasoning can be resolved through the use of advanced probability theory. A Bayesian Network Model however, not only provides a structured reasoning approach, but more importantly, it can also resolve the circular reasoning problem that has been identified. / Captain, Singapore Army

Child protective services begin with an intake (screening) decision to accept or reject maltreatment reports. This crucial decision may lead to significant positive or negative outcomes for children and families. Little is known about characteristics that intake decision-makers share or factors that influence the decision-making process. Racially-biased intake practices have been blamed for contributing to African American children’s disproportionate overrepresentation in the child welfare system. Concerns have emerged that social workers may hold negative stereotypes about African Americans and parents who use drugs. Stereotypical biases may influence decisions in reports alleging parental drug use and/or involving African American families. This study was conducted to examine the influence of race and parental drug-use allegations on intake decision-making. It was also conducted to identify factors that influence decision-making and to determine whether concepts drawn from naturalistic decision theory and attribution theory are relevant to intake decision-making. A conceptual model for describing decision-making was proposed and tested. Equivalent materials design was employed. Respondents completed an on-line questionnaire that included 24 vignettes describing hypothetical maltreatment concerns. Race and drug use were manipulated between two instrument versions. Respondents completed a 45-item scale measuring racial and parental drug use bias. They also described their application of policy to decision-making and the degree to which they engaged in different types of mental simulation (a naturalistic decision theory strategy) in making decisions. Eighty-seven child protective services intake decision-makers in Virginia participated (67% response rate). The findings suggest that respondents’ decisions were not influenced by racial bias but were influenced by parental drug use bias. Respondents’ parental drug use bias scores were higher than their racial bias scores. Social workers’ racial bias scores were higher than other respondents’ scores. A set of nine primary decision-factors used frequently in decision-making was identified. Finally, respondents reported using their discretion in adhering to CPS policy depending upon their concern for children’s safety. The research contributes to understanding the intake decision-making process. Findings related to worker characteristics, relevant decision-factors, and decision-making behaviors may influence practice and future research. Findings also suggest that naturalistic decision theory concepts warrant further attention and study.

This thesis takes three different perspectives, using theoretical and experimental techniques, on time-inconsistent preferences and how the existence of multiple selves can affect both consumer behaviour and policy design. Across domains such as retirement saving, health, and educational achievement, intertemporal choice presents a challenge for both individuals and policymakers. The first paper, 'Choosing When to Nudge: Designing Behavioural Policy around Decision-Making Costs,' considers how behavioural policy, which has proven increasingly popular with policymakers, affects welfare. I find that for present-biased consumers, behavioural policies help some consumers but can inefficiently discourage others from optimizing. Such policies therefore have an ambiguous effect on welfare, and similar to traditional policies, can impose equity-efficiency tradeoffs. Monopolies may increase welfare given their incentive to simplify consumer decisions instead of exploit switching costs. The second paper, 'Virtue and Vice with Endogenous Preferences,' considers behaviour when preferences are affected by consumer decisions. I introduce agents whose temptation to consume in the present is affected by how much they choose to save for the future. I find that differences between agents can trap them in divergent paths of self-improvement -- saving more, they value the future more, making saving optimal -- or binging -- consuming more makes them indifferent to future costs, making consumption optimal. At the extreme, it is frequently an optimum for a consumer to consume their entire wealth. The final paper, 'Bet You Can't Eat Just One: Consumption Complementarity and 'Self'-Control' considers an intrapersonal game between a moderate cold self and a hot self that wants to indulge. In equilibrium, sophisticated selves best respond to each other's behaviour: the cold self over-abstains and the hot self over-indulges to avoid inducing the other state. I test these ideas in the lab, and find that subjects on a diet who were induced to consume a piece of chocolate before the experiment indulge more in chocolate during the experiment, even when the initial indulgence was imposed by the experimenter. Eating a piece of chocolate, this suggests, can induce a period during which chocolate is more appealing.

Le domaine abordé dans notre travail se situe autour de l'aide à la décision multi-critère. Nous avons tout d'abord étudié les risques bancaires au travers d'une revue de la littérature très large. Nous avons ensuite élaboré un modèle théorique regroupant quatre catégories différentes composées de dix-neuf cas de risques financiers. Au travers de ce modèle théorique, nous avons mené une validation expérimentale avec un cas réel : La caisse d'épargne du Midi-Pyrénées. Cette validation expérimentale nous a permis un approfondissement des analyses des pratiques pour la gestion des risques financières. Dans cette thèse, nous cherchons à apporter une contribution à la gestion des risques dans le domaine du secteur financier et plus particulièrement pour la sécurité de système d'information et plus précisément au niveau de la caisse d'épargne. Ces analyses s'appuient donc sur des faits observés, recueillis et mesurés, des expérimentations réelles, résultant de la politique de sécurité des systèmes d'information et voulant offrir une approche pragmatique de la présentation de l'analyse de risques financiers grâce à des méthodes d'aide multicritère à la décision. L'élaboration de ce modèle permet de représenter certains aspects spécifiques des risques financiers. Nos recherches ont donné lieu à la réalisation d'un résultat concret : un système d'aide à la décision pour les besoins du responsable du système d'information de la caisse d'épargne. Il s'agit d'un système efficace présentant les résultats sous forme de figures relatives pour les valeurs des critères attribués par le responsable de système de sécurité d'informations (RSSI). / We are working on multicriteria decision analysis. We started with the study of risk typology through a huge review of literature. We have developed a theoretical model grouping four different categories of nineteen financial risk cases. Through this theoretical model, we have applied them to the "Caisse d'Epargne Midi-Pyrénées". In this thesis, we seek to make a contribution to the security management of information systems at the level of the savings bank. These analyzes are based on facts observed, collected and measured with real experiments resulting in its information system security policy and want to offer a pragmatic approach to the presentation of financial risk analysis through methods supporting. multicriteria decision analysis. The development of this model makes it possible to represent certain specific aspects of the financial risks that have often occurred in their activities. Our research led to the achievement of a concrete result in relation to the needs of the information system manager of the savings bank. It is an effective decision support system by constructing relative figures for the values of the criteria assigned by the RSSI.

[pt] Muitos problemas de transportes, assim como de outras áreas do conhecimento, envolvem tomada de decisão. Em decisões complexas, a escolha da melhor alternativa ou plano de ação pode envolver mais de um critério e é necessário estudar como cada ação afeta cada critério. O método AHP, Analytic Hierarchy Process, proposto por Thomas L. Saaty, é um método de decisão multicriterial que funciona para os mais diversos tipos de decisões, solucionando problemas com fatores quantitativos e qualitativos. Ele reúne a opinião dos tomadores de decisão em matrizes de comparação. Este trabalho faz uma revisão geral de conceitos básicos do método, mostrando diferentes maneiras de cálculo da solução. A primeira explorada é o cálculo exato através dos autovalores e autovetores das matrizes. Para esse cálculo, foi utilizado o software francês Scilab, semelhante ao mais conhecido Matlab, mas distibuído gratuitamente na internet. É discutida a questão da consistência dos julgamentos, com maneiras de medi-la e melhorá-la. Finalmente, é feita uma proposta de solução aproximada, que questiona a idéia original de que um certo nível de inconsistência é desejável. É uma solução simplificada que, supondo consistência absoluta, facilita não só os cálculos como o trabalho inicial dos tomadores de decisão. Em vez de comparar todas as alternativas com as outras, duas a duas, passa a ser necessário comparar apenas uma alternativa com as outras. A nova solução aproximada é comparada com a solução exata em três casos retirados da literatura. / [en] Several transportation problems, as well as problems in other knowledge areas, request decision making. In complex decisions, the choice of best alternative or course of action can contain more than one criterion and it is necessary to study how each alternative affects each criterion. The AHP, Analytic Hierarchy Process, proposed by Thomas L. Saaty, is a multicriteria decision method that works well for very diverse decision types, solving problems with tangible and intangible factors. It gathers the opinion of decision makers in comparison matrices. This study makes a general review of basic concepts of the method, showing different manners of calculating the solution. The first one to be displayed is the exact solution using the eigenvalues and eigenvectors of the matrices. For this solution the French software Scilab was used, which is similar to the well-known Matlab, but free and distributed on the web. The issue of judgment consistency is discussed, including ways of measuring and improving it. Finally, a proposal of approximated solution is made, questioning the original idea which says that a certain level of inconsistency is desirable. It is a simplification that, considering absolute consistency, facilitates not only the calculations but also the early work of decision makers when judging the alternatives. Instead of making pair wise comparisons of all alternatives with each other, it becomes necessary to compare only one alternative with the others. The new approximated solution is compared to the real solution in three cases taken from the literature.

The focus of this thesis is to statistically model violent crime rates against population over the years 1960-2009 for the United States. We approach this question as to be of interest since the trend of population for individual states follows different patterns. We propose here a method which employs cubic spline regression modeling. First we introduce a minimum/maximum algorithm that will identify potential knots. Then we employ least squares estimation to find potential regression coefficients based upon the cubic spline model and the knots chosen by the minimum/maximum algorithm. We then utilize the best subsets regression method to aid in model selection in which we find the minimum value of the Bayesian Information Criteria. Finally, we preent the R2adj as a measure of overall goodness of fit of our selected model. We have found among the fifty states and Washington D.C., 42 out of 51 showed an R2adj value that was greater than 90%. We also present an overall model of the United States. Also, we show additional applications our algorithm for data which show a non linear association. It is hoped that our method can serve as a unified model for violent crime rate over future years. / by Eric David Golinko. / Thesis (M.S.)--Florida Atlantic University, 2012. / Includes bibliography. / Electronic reproduction. Boca Raton, Fla., 2012. Mode of access: World Wide Web.