Mathematical modeling has been instrumental to the development of natural sciences over the last half-century. Through iterated interactions between modeling and real-world exper- imentation, these models have furthered our understanding of the processes in biology and chemistry that they seek to represent. In certain application domains, such as the field of car- diac biology, communities of modelers with common interests have emerged, leading to the development of many models that attempt to explain the same or similar phenomena. As these communities have developed, however, reporting standards for modeling studies have been in- consistent, often focusing on the final parameterized result, and obscuring the assumptions and data used during their creation. These practices make it difficult for researchers to adapt exist- ing models to new systems or newly available data, and also to assess the identifiability of said models - the degree to which their optimal parameters are constrained by data - which is a key step in building trust that their formulation captures truth about the system of study. In this thesis, we develop tools that allow modelers working with biological or chemical time series data to assess identifiability in an automated fashion, and embed these tools within a novel online community resource that enforces reproducible standards of reporting and facilitates exchange of models and data. We begin by exploring the application of Bayesian and approximate Bayesian inference methods, which parameterize models while simultaneously assessing uncertainty about these estimates, to assess the identifiability of models of the cardiac action potential. We then demon- strate how the side-by-side application of these Bayesian and approximate Bayesian methods can be used to assess the information content of experiments where system observability is limited to "summary statistics" - low-dimensional representations of full time-series data. We next investigate how a posteriori methods of identifiability assessment, such as the above inference techniques, compare against a priori methods based on model structure. We compare these two approaches over a range of biologically relevant experimental conditions, and high- light the cases under which each strategy is preferable. We also explore the concept of optimal experimental design, in which measurements are chosen in order to maximize model identifia- bility, and compare the feasibility of established a priori approaches against a novel a posteriori approach. Finally, we propose a framework for representing and executing modeling experiments in a reproducible manner, and use this as the foundation for a prototype "Modeling Web Lab" where researchers may upload specifications for and share the results of the types of inference explored in this thesis. We demonstrate the Modeling Web Lab's utility across multiple mod- eling domains by re-creating the results of a contemporary modeling study of the hERG ion channel model, as well as the results of an original study of electrochemical redox reactions. We hope that this works serves to highlight the importance of both reproducible standards of model reporting, as well as identifiability assessment, which are inherently linked by the desire to foster trust in community-developed models in disciplines across the natural sciences.

This study demonstrates a fully Bayesian approach to multilevel/hierarchical linear growth modeling using freely available software. Further, the study incorporates informative prior distributions for fixed effect estimates using an objective approach. The objective approach uses previous sample results to form prior distributions included in subsequent samples analyses, a process referred to as Bayesian updating. Further, a method for model checking is outlined based on fit indices including information criteria (i.e., Akaike information criterion, Bayesian information criterion, and deviance information criterion) and approximate Bayes factor calculations. For this demonstration, five distinct samples of schools in the process of implementing School-Wide Positive Behavior Interventions and Supports (SWPBIS) collected from 2008 to 2013 were used with the unit of analysis being the school. First, the within-year SWPBIS fidelity growth was modeled as a function of time measured in months from initial measurement occasion. Uninformative priors were used to estimate growth parameters for the 2008-09 sample, and both uninformative and informative priors based on previous years' samples were used to model data from the 2009-10, 2010-11, 2011-12, 2012-13 samples. Bayesian estimates were also compared to maximum likelihood (ML) estimates, and reliability information is provided. Second, an additional three examples demonstrated how to include predictors into the growth model with demonstrations for: (a) the inclusion of one school-level predictor (years implementing) of SWPBIS fidelity growth, (b) several school-level predictors (relative socio-economic status, size, and geographic location), and (c) school and district predictors (sustainability factors hypothesized to be related to implementation processes) in a three-level growth model. Interestingly, Bayesian models estimated with informative prior distributions in all cases resulted in more optimal fit indices than models estimated with uninformative prior distributions.

L'imagerie ultrasonore est une modalité d'acquisition privilégiée en imagerie médicale en raison de son innocuité, sa simplicité d'utilisation et son coût modéré d'utilisation. Néanmoins, la résolution limitée et le faible contraste limitent son utilisation dans certaines d'applications. C'est dans ce contexte que différentes techniques de post-traitement visant à améliorer la qualité de telles images sont proposées dans ce manuscrit. Dans un premier temps, nous proposons d'aborder le problème conjoint de la déconvolution et de la segmentation d'images ultrasonores en exploitant l'interaction entre ces deux problèmes. Le problème, énoncé dans un cadre bayésien, est résolu à l'aide d'un algorithme MCMC en raison de la complexité de la loi a posteriori des paramètres d'intérêt. Dans un second temps, nous proposons une nouvelle méthode rapide de super-résolution fondée sur la résolution analytique d'un problème de minimisation l2-l2. Il convient de remarquer que les deux approches proposées peuvent être appliquées aussi bien à des images ultrasonores qu'à des images naturelles ou constantes par morceaux. Enfin, nous proposons une méthode de déconvolution aveugle basée sur un modèle paramétrique de la réponse impulsionelle de l'instrument ou du noyau de flou. / In the field of medical image analysis, ultrasound is a core imaging modality employed due to its real time and easy-to-use nature, its non-ionizing and low cost characteristics. Ultrasound imaging is used in numerous clinical applications, such as fetus monitoring, diagnosis of cardiac diseases, flow estimation, etc. Classical applications in ultrasound imaging involve tissue characterization, tissue motion estimation or image quality enhancement (contrast, resolution, signal to noise ratio). However, one of the major problems with ultrasound images, is the presence of noise, having the form of a granular pattern, called speckle. The speckle noise in ultrasound images leads to the relative poor image qualities compared with other medical image modalities, which limits the applications of medical ultrasound imaging. In order to better understand and analyze ultrasound images, several device-based techniques have been developed during last 20 years. The object of this PhD thesis is to propose new image processing methods allowing us to improve ultrasound image quality using postprocessing techniques. First, we propose a Bayesian method for joint deconvolution and segmentation of ultrasound images based on their tight relationship. The problem is formulated as an inverse problem that is solved within a Bayesian framework. Due to the intractability of the posterior distribution associated with the proposed Bayesian model, we investigate a Markov chain Monte Carlo (MCMC) technique which generates samples distributed according to the posterior and use these samples to build estimators of the ultrasound image. In a second step, we propose a fast single image super-resolution framework using a new analytical solution to the l2-l2 problems (i.e., $\ell_2$-norm regularized quadratic problems), which is applicable for both medical ultrasound images and piecewise/ natural images. In a third step, blind deconvolution of ultrasound images is studied by considering the following two strategies: i) A Gaussian prior for the PSF is proposed in a Bayesian framework. ii) An alternating optimization method is explored for blind deconvolution of ultrasound.

abstract: Many methodological approaches have been utilized to predict student retention and persistence over the years, yet few have utilized a Bayesian framework. It is believed this is due in part to the absence of an established process for guiding educational researchers reared in a frequentist perspective into the realms of Bayesian analysis and educational data mining. The current study aimed to address this by providing a model-building process for developing a Bayesian network (BN) that leveraged educational data mining, Bayesian analysis, and traditional iterative model-building techniques in order to predict whether community college students will stop out at the completion of each of their first six terms. The study utilized exploratory and confirmatory techniques to reduce an initial pool of more than 50 potential predictor variables to a parsimonious final BN with only four predictor variables. The average in-sample classification accuracy rate for the model was 80% (Cohen's κ = 53%). The model was shown to be generalizable across samples with an average out-of-sample classification accuracy rate of 78% (Cohen's κ = 49%). The classification rates for the BN were also found to be superior to the classification rates produced by an analog frequentist discrete-time survival analysis model. / Dissertation/Thesis / Doctoral Dissertation Educational Psychology 2015

Social learning is ubiquitous among animals and humans and is thought to be critical to the widespread success of humans and to the development and evolution of human culture. Evolutionary theory, however, suggests that social learning alone may not be adaptive but that individuals may need to be selective in who and how they copy others. One of the key findings of these evolutionary models (reviewed in Chapter 1) is that social information may be widely adaptive if individuals are able to combine social and asocial sources of information together strategically. However, up until this point the focus of theoretic models has been on the population level consequences of different social learning strategies, and not on how individuals combine social and asocial information on specific tasks. In Chapter 2 I carry out an analysis of how animal learners might incorporate social information into a reinforcement learning framework and find that even limited, low-fidelity copying of actions in an action sequence may combine with asocial learning to result in high fidelity transmission of entire action sequences. In Chapter 3 I describe a series of experiments that find that human learners flexibly use a conformity biased learning strategy to learn from multiple demonstrators depending on demonstrator accuracy, either indicated by environmental cues or past experience with these demonstrators. The chapter reveals close quantitative and qualitative matches between participant's performance and a Bayesian model of social learning. In both Chapters 2 and 3 I find, consistent with previous evolutionary findings, that by combining social and asocial sources of information together individuals are able to learn about the world effectively. Exploring how animals use social learning experimentally can be a substantially more difficult task than exploring human social learning. In Chapter 4, I develop and present a refined version of Network Based Diffusion analysis to provide a statistical framework for inferring social learning mechanisms from animal diffusion experiments. In Chapter 5 I move from examining the effects of social learning at an individual level to examining their population level outcomes and provide an analysis of how fine-grained population structure may alter the spread of novel behaviours through a population. I find that although a learner's social learning strategy and the learnability of a novel behaviour strongly impact how likely the behaviour is to spread through the population, fine grained population structure plays a much smaller role. In Chapter 6 I summarize the results of this thesis, and provide suggestions for future work to understand how individuals, humans and other animals alike, use social information.

A pivotal moment in machine learning research was the creation of an important new research area, known as Ensemble Learning. In this work, we argue that ensembles are a very general concept, and though they have been widely used, they can be applied in more situations than they have been to date. Rather than using them only to combine the output of an algorithm, we can apply them to decisions made inside the algorithm itself, during the learning step. We call this approach Inner Ensembles. The motivation to develop Inner Ensembles was the opportunity to produce models with the similar advantages as regular ensembles, accuracy and stability for example, plus additional advantages such as comprehensibility, simplicity, rapid classification and small memory footprint. The main contribution of this work is to demonstrate how broadly this idea can be applied, and highlight its potential impact on all types of algorithms. To support our claim, we first provide a general guideline for applying Inner Ensembles to different algorithms. Then, using this framework, we apply them to two categories of learning methods: supervised and un-supervised. For the former we chose Bayesian network, and for the latter K-Means clustering. Our results show that 1) the overall performance of Inner Ensembles is significantly better than the original methods, and 2) Inner Ensembles provide similar performance improvements as regular ensembles.

Dendrocolaptes platyrostris é uma espécie de ave florestal associada às matas de galeria do corredor de vegetação aberta da América do sul (D. p. intermedius) e à Floresta Atlântica (D. p. platyrostris). Em um trabalho anterior, foi observada estrutura genética populacional associada às subespécies, além de dois clados dentro da Floresta Atlântica e evidências de expansão na população do sul, o que é compatível com o modelo Carnaval-Moritz. Utilizando approximate Bayesian computation, o presente trabalho avaliou a diversidade genética de dois marcadores nucleares e um marcador mitocondrial dessa espécie com o objetivo de comparar os resultados obtidos anteriormente com os obtidos utilizando uma estratégia multi-locus e considerando variação coalescente. Os resultados obtidos sugerem uma relação de politomia entre as populações que se separaram durante o último período interglacial, mas expandiram após o último máximo glacial. Este resultado é consistente com o modelo de Carnaval-Moritz, o qual sugere que as populações sofreram alterações demográficas devido às alterações climáticas ocorridas nestes períodos. Trabalhos futuros incluindo outros marcadores e modelos que incluam estabilidade em algumas populações e expansão em outras são necessários para avaliar o presente resultado / Dendrocolaptes platyrostris is a forest specialist bird associated to gallery forests of the open vegetation corridor of South America (D. p. intermedius) and to the Atlantic forest (D. p. platyrostris). A previous study showed a population genetic structure associated with the subspecies, two clades within the Atlantic forest, and evidence of population expansion in the south, which is compatible with Carnaval- Moritz\'s model. The present study evaluated the genetic diversity of two nuclear and one mitochondrial markers of this species using approximate Bayesian computation, in order to compare the results previously obtained with those based on a multi-locus strategy and considering the coalescent variation. The results suggest a polytomic relationship among the populations that split during the last interglacial period and expanded after the last glacial maximum. This result is consistent with the model of Carnaval-Moritz, which suggests that populations have undergone demographic changes due to climatic changes that occurred in these periods. Future studies including other markers and models that include stability in some populations and expansion in others are needed to evaluate the present result

Direct marketing problems have been intensively reviewed in the marketing literature recently, such as purchase frequency and time, sales profit, and brand choices. However, modeling the customer response, which is an important issue in direct marketing research, remains a significant challenge. This thesis is an empirical study of predicting customer response to direct marketing and applies a Bayesian approach, including the Bayesian Binary Regression (BBR) and the Hierarchical Bayes (HB). Other classical methods, such as Logistic Regression and Latent Class Analysis (LCA), have been conducted for the purpose of comparison. The results of comparing the performance of all these techniques suggest that the Bayesian methods are more appropriate in predicting direct marketing customer responses. Specifically, when customers are analyzed as a whole group, the Bayesian Binary Regression (BBR) has greater predictive accuracy than Logistic Regression. When we consider customer heterogeneity, the Hierarchical Bayes (HB) models, which use demographic and geographic variables for clustering, do not match the performance of Latent Class Analysis (LCA). Further analyses indicate that when latent variables are used for clustering, the Hierarchical Bayes (HB) approach has the highest predictive accuracy.

Supply chain management (SCM) is the oversight of materials, information, and finances as they move in a process from supplier to manufacturer to wholesaler to retailer to consumer. Supply chain management involves coordinating and integrating these flows both within and among companies as efficiently as possible. The supply chain consists of interconnected components that can be complex and dynamic in nature. Therefore, an interruption in one subnetwork of the system may have an adverse effect on another subnetworks, which will result in a supply chain disruption. Disruptions from an event or series of events can have costly and widespread ramifications. When a disruption occurs, the speed at which the problem is discovered becomes critical. There is an urgent need for efficient monitoring of the supply chain. By examining the vulnerability of the supply chain network, supply chain managers will be able to mitigate risk and develop quick response strategies in order to reduce supply chain disruption. However, modeling these complex supply chain systems is a challenging research task. This research is concerned with developing an extended Bayesian Network approach to analyze supply chain disruptions. The aim is to develop strategies that can reduce the adverse effects of the disruptions and hence improve overall system reliability. The supply chain disruptions is modeled using Bayesian Networks-a method of modeling the cause of current and future events, which has the ability to model the large number of variables in a supply chain and has proven to be a powerful tool under conditions of uncertainty. Two impact factors are defined. These are the Bayesian Impact Factor (BIF) and the Node Failure Impact Factor (NFIF). An industrial example is used to illustrate the application proposed to make the supply chain more reliable.

abstract: Information exists in various forms and a better utilization of the available information can benefit the system awareness and response predictions. The focus of this dissertation is on the fusion of different types of information using Bayesian-Entropy method. The Maximum Entropy method in information theory introduces a unique way of handling information in the form of constraints. The Bayesian-Entropy (BE) principle is proposed to integrate the Bayes’ theorem and Maximum Entropy method to encode extra information. The posterior distribution in Bayesian-Entropy method has a Bayesian part to handle point observation data, and an Entropy part that encodes constraints, such as statistical moment information, range information and general function between variables. The proposed method is then extended to its network format as Bayesian Entropy Network (BEN), which serves as a generalized information fusion tool for diagnostics, prognostics, and surrogate modeling. The proposed BEN is demonstrated and validated with extensive engineering applications. The BEN method is first demonstrated for diagnostics of gas pipelines and metal/composite plates for damage diagnostics. Both empirical knowledge and physics model are integrated with direct observations to improve the accuracy for diagnostics and to reduce the training samples. Next, the BEN is demonstrated in prognostics and safety assessment in air traffic management system. Various information types, such as human concepts, variable correlation functions, physical constraints, and tendency data, are fused in BEN to enhance the safety assessment and risk prediction in the National Airspace System (NAS). Following this, the BE principle is applied in surrogate modeling. Multiple algorithms are proposed based on different type of information encoding, such as Bayesian-Entropy Linear Regression (BELR), Bayesian-Entropy Semiparametric Gaussian Process (BESGP), and Bayesian-Entropy Gaussian Process (BEGP) are demonstrated with numerical toy problems and practical engineering analysis. The results show that the major benefits are the superior prediction/extrapolation performance and significant reduction of training samples by using additional physics/knowledge as constraints. The proposed BEN offers a systematic and rigorous way to incorporate various information sources. Several major conclusions are drawn based on the proposed study. / Dissertation/Thesis / Doctoral Dissertation Mechanical Engineering 2020