Model discrimination deals with situations where there are several candidate models available to represent a system. The objective is to find the “best” model among rival models with respect to prediction of system behavior. Empirical and mechanistic models are two important categories of models. Mechanistic models are developed based on physical mechanisms. These types of models can be applied for prediction purposes, but they are also developed to gain improved understanding of the underlying physical mechanism or to estimate physico-chemical parameters of interest. When model discrimination is applied to mechanistic models, the main goal is typically to determine the “correct” underlying physical mechanism. This study focuses on mechanistic models and presents a model discrimination procedure which is applicable to mechanistic models for the purpose of studying the underlying physical mechanism. Obtaining the data needed from the real system is one of the challenges particularly in applications where experiments are expensive or time consuming. Therefore, it is beneficial to get the maximum information possible from the real system using the least possible number of experiments. In this research a new approach to model discrimination is presented that takes advantage of Monte Carlo (MC) methods. It combines a design of experiments (DOE) method with an adaptation of MC model selection methods to obtain a sequential Bayesian Markov Chain Monte Carlo model discrimination framework which is general and usable for a wide range of model discrimination problems. The procedure has been applied to chemical engineering case studies and the promising results have been discussed. Four case studies, order of reaction, rate of FeIII formation, copolymerization, and RAFT polymerization, are presented in this study. The first three benchmark problems allowed us to refine the proposed approach. Moreover, applying the Sequential Bayesian Monte Carlo model discrimination framework in the RAFT problem made a contribution to the polymer community by recommending analysis an approach to selecting the correct mechanism.

This paper is concerned with the problem of finding an experimental design for discrimination between two rival models and for model robustness that minimizing the maximum bias simultaneously in binary response experiments. The criterion for model discrimination is based on the $T$-optimality criterion proposed in Atkinson and Fedorov (1975), which maximizes the sum of squares of deviations between the two rival models while the criterion for model robustness is based on minimizing the maximum probability bias of the two rival models. In this paper we obtain the optimum designs satisfy the above two criteria for some commonly used rival models in binary response experiments such as the probit and logit models etc.

It is suggested to discriminate between different state space models for a given time series by means of a Bayesian approach which chooses the model that minimizes the expected loss. Practical implementation of this procedures requires a fully Bayesian analysis for both the state vector and the unknown hyperparameters which is carried out by Markov chain Monte Carlo methods. Application to some non-standard situations such as testing hypotheses on the boundary of the parameter space, discriminating non-nested models and discrimination of more than two models is discussed in detail. (author's abstract) / Series: Forschungsberichte / Institut für Statistik

Model discrimination deals with situations where there are several candidate models available to represent a system. The objective is to find the “best” model among rival models with respect to prediction of system behavior. Empirical and mechanistic models are two important categories of models. Mechanistic models are developed based on physical mechanisms. These types of models can be applied for prediction purposes, but they are also developed to gain improved understanding of the underlying physical mechanism or to estimate physico-chemical parameters of interest. When model discrimination is applied to mechanistic models, the main goal is typically to determine the “correct” underlying physical mechanism. This study focuses on mechanistic models and presents a model discrimination procedure which is applicable to mechanistic models for the purpose of studying the underlying physical mechanism. Obtaining the data needed from the real system is one of the challenges particularly in applications where experiments are expensive or time consuming. Therefore, it is beneficial to get the maximum information possible from the real system using the least possible number of experiments. In this research a new approach to model discrimination is presented that takes advantage of Monte Carlo (MC) methods. It combines a design of experiments (DOE) method with an adaptation of MC model selection methods to obtain a sequential Bayesian Markov Chain Monte Carlo model discrimination framework which is general and usable for a wide range of model discrimination problems. The procedure has been applied to chemical engineering case studies and the promising results have been discussed. Four case studies, order of reaction, rate of FeIII formation, copolymerization, and RAFT polymerization, are presented in this study. The first three benchmark problems allowed us to refine the proposed approach. Moreover, applying the Sequential Bayesian Monte Carlo model discrimination framework in the RAFT problem made a contribution to the polymer community by recommending analysis an approach to selecting the correct mechanism.

La détermination des cinétiques réactionnelles est incontournable en génie de la réaction. Sans quoi on ne peut dimensionner convenablement un réacteur. Cependant, la complexité de certains systèmes réactionnels nécessite de fournir des efforts expérimentaux souvent rédhibitoires, en termes de moyens comme de temps. Des méthodologies de planification expérimentales pour la modélisation cinétique existent. Les objectifs de ces méthodes peuvent être la détermination précise des paramètres d’un modèle ou la discrimination de plusieurs modèles potentiels. Notre objectif est d’étudier des méthodologies pour discriminer entre des modèles et d’éprouver ces méthodologies sur des applications numériques et/ou expérimentales. Ces méthodologies s’appuient sur un processus itératif, qui, étape par étape, mène à la sélection d’un modèle, à la détermination de ses paramètres et à sa validation. Une fois la méthodologie précisée, un premier cas d’étude « numérique » concerne un système réactionnel, dont plusieurs modèles sont proposés, basés sur différents schémas réactionnels. L’objectif étant de trouver le bon schéma par application de la méthodologie. Ensuite un cas d’étude expérimentale est traité : l’acylation catalytique du ferrocène. Sa cinétique n’est pas connue. La méthodologie appliquée mènera à la proposition de plusieurs modèles cinétiques et à la sélection du plus adapté. Un dernier cas d’étude s’intéresse à la modélisation des réactions exothermique, en particulier en micro/milli réacteur continu. Un focus est d’abord fait sur une modélisation adaptée à ces systèmes réactionnels, à travers l’utilisation de nombres adimensionnels réduisant les degrés de liberté, et permettant une analyse synthétique du comportement du réacteur. Par la suite, plusieurs applications « numériques » de la méthodologie de discrimination de modèles sont proposés, dont l’objectif est la discrimination du comportement thermique des réacteurs. La méthodologie expérimentale utilise efficacement les données déjà accumulées sur les modèles, pour choisir au mieux chaque nouvelle expérience en fonction des objectifs ciblés. Elle permet de cibler efficacement les meilleures expériences pour atteindre l’objectif fixé. Les efforts expérimentaux sont ainsi optimisés, ainsi que la recherche de modèles cinétiques et thermiques. / Knowledge on kinetics is essential for chemical reactor modelling. Yet when chemical systems are very complex, development of good kinetic models leads to expensive and time consuming experiments, often prohibitive. Our goal is to develop efficient numerical methods to design the optimal experiments to select the best model among many possible candidates while precisely estimating its kinetics parameters. The gain is double: reduction of experiments and acquisition of more accurate information. Several study cases will enable the assessment of these methodologies. Optimal experimental design methodologies are based on iterative procedures,leading to the selection of the more accurate model, and to the identification of its parameters. In the present work, a first numerical study case is chosen as an illustration of the method, consisting on the discrimination of four synthesis pathway that are potential candidates to describe a reactional system. It is showed how the developed method can smartly choose experiments to lead to the choice of the accurate pathway. The second study case is the experimental study of the catalytic acetylation of ferrocene, for which any accurate kinetic models have not been found yet. Thanks to the iterative design of experiments, it is possible to characterize, very quickly, the order of reaction and how the catalyst effect has to be considered. The last part of this work deals with exothermic reactions and the coupling between thermal transfer and chemical reactions in milli/micro-reactors. The use of dimensionless numbers is proposed to reduce the number of parameters implied in such systems and to analyse the thermal behaviour of microreactors. Then, the aim is to illustrate how to discriminate thermal behaviours using the discriminatory methodology, through several study cases. Those examples demonstrate that iterative design of experiments is an efficient method to find the best experiments to solve the issues for selecting a model among others and for determining the associated parameters. This offers the advantages to reduce the experimental efforts in time and in matter, and thus to unlock modelling of many complex chemical systems.

abstract: This work considers the design of separating input signals in order to discriminate among a finite number of uncertain nonlinear models. Each nonlinear model corresponds to a system operating mode, unobserved intents of other drivers or robots, or to fault types or attack strategies, etc., and the separating inputs are designed such that the output trajectories of all the nonlinear models are guaranteed to be distinguishable from each other under any realization of uncertainties in the initial condition, model discrepancies or noise. I propose a two-step approach. First, using an optimization-based approach, we over-approximate nonlinear dynamics by uncertain affine models, as abstractions that preserve all its system behaviors such that any discrimination guarantees for the affine abstraction also hold for the original nonlinear system. Then, I propose a novel solution in the form of a mixed-integer linear program (MILP) to the active model discrimination problem for uncertain affine models, which includes the affine abstraction and thus, the nonlinear models. Finally, I demonstrate the effectiveness of our approach for identifying the intention of other vehicles in a highway lane changing scenario. For the abstraction, I explore two approaches. In the first approach, I construct the bounding planes using a Mixed-Integer Nonlinear Problem (MINLP) formulation of the given system with appropriately designed constraints. For the second approach, I solve a linear programming (LP) problem that over-approximates the nonlinear function at only the grid points of a mesh with a given resolution and then accounting for the entire domain via an appropriate correction term. To achieve a desired approximation accuracy, we also iteratively subdivide the domain into subregions. This method applies to nonlinear functions with different degrees of smoothness, including Lipschitz continuous functions, and improves on existing approaches by enabling the use of tighter bounds. Finally, we compare the effectiveness of this approach with the existing optimization-based methods in simulation and illustrate its applicability for estimator design. / Dissertation/Thesis / Masters Thesis Mechanical Engineering 2018

This thesis examines the effect of foreign nationality on the outcomes of criminal process in the Czech Republic. Foreign citizens are overrepresented by 2% compared to their share in population in all stages of the criminal process, suggesting possible discrimination by domestic authorities. Using rich case level datasets from 2005 to 2015 observed gaps are decomposed to part explained by a difference in the objective characteristics of the cases and unexplained part, suggesting possible inequality of treatment. Foreigner gaps in probability of charge, probability of conviction, probability of imprisonment, length of the sentence and probability of release from prison on parole are decomposed. Majority of observed disparities in the outcomes can be attributed to higher involvement of foreigners in more serious crimes compared to nationals. Unexplained disparities remain in probability of imprisonment and probability of release on parole, suggesting possible unequal treatment in these two outcomes.

abstract: In this dissertation two research questions in the field of applied experimental design were explored. First, methods for augmenting the three-level screening designs called Definitive Screening Designs (DSDs) were investigated. Second, schemes for strategic subdata selection for nonparametric predictive modeling with big data were developed. Under sparsity, the structure of DSDs can allow for the screening and optimization of a system in one step, but in non-sparse situations estimation of second-order models requires augmentation of the DSD. In this work, augmentation strategies for DSDs were considered, given the assumption that the correct form of the model for the response of interest is quadratic. Series of augmented designs were constructed and explored, and power calculations, model-robustness criteria, model-discrimination criteria, and simulation study results were used to identify the number of augmented runs necessary for (1) effectively identifying active model effects, and (2) precisely predicting a response of interest. When the goal is identification of active effects, it is shown that supersaturated designs are sufficient; when the goal is prediction, it is shown that little is gained by augmenting beyond the design that is saturated for the full quadratic model. Surprisingly, augmentation strategies based on the I-optimality criterion do not lead to better predictions than strategies based on the D-optimality criterion. Computational limitations can render standard statistical methods infeasible in the face of massive datasets, necessitating subsampling strategies. In the big data context, the primary objective is often prediction but the correct form of the model for the response of interest is likely unknown. Here, two new methods of subdata selection were proposed. The first is based on clustering, the second is based on space-filling designs, and both are free from model assumptions. The performance of the proposed methods was explored visually via low-dimensional simulated examples; via real data applications; and via large simulation studies. In all cases the proposed methods were compared to existing, widely used subdata selection methods. The conditions under which the proposed methods provide advantages over standard subdata selection strategies were identified. / Dissertation/Thesis / Doctoral Dissertation Statistics 2020

Model reliability is important when groundwater models are used for evaluation of environmental impact and water resource management. Model attributes such as geohydrologic units and parameter values need to be quantified in order to obtain reliable results. A primary objective of sampling design for groundwater models is to increase the reliability of modelling results by selecting effective measurement locations and times. It is advantageous to employ simulation models to guide measurement strategies already in early investigation stages. Normally, optimal design is only possible when model attributes are known prior to constructing a design. This is not a meaningful requirement as the model attributes are the final result of the analysis and are not known beforehand. Thus, robust design methods are required that are effective for ranges of parameter values, measurement error types and for alternative conceptual models. Parameter sensitivity is the fundamental model property that is used in this thesis to create effective designs. For conceptual model uncertainty, large-scale sensitivity analysis is used to devise networks that capture sufficient information to determine which model best describes the system with a minimum of measurement points. In fixed conceptual models, effective parameter- and error-robust designs are based on criteria that minimise the size of the parameter covariance matrix (D-optimality). Optimal designs do not necessarily have observations with the highest parameter sensitivities because D-optimality reduces parameter estimation errors by balancing high sensitivity and low correlation between parameters. Ignoring correlation in sparse designs may result in considerably inefficient designs. Different measurement error assumptions may also give widely different optimal designs. Early stage design often involves simple homogenous models for which the design effectiveness may be seriously offset by significant aquifer heterogeneity. Simple automatic and manual methods are possible for design generation. While none of these guarantee globally optimal designs, they do generate designs that are more effective than those normally used for measurement programs. Effective designs are seldom intuitively obvious, indicating that this methodology is quite useful. A general benefit of this type of analysis, in addition to the actual generation of designs, is insight into the relative importance of model attributes and their relation to different measurement strategies.

The thesis focuses on the development of statistical theory in experimental design with applications in global optimization. It consists of four parts. In the first part, a criterion of design efficiency, under model uncertainty, is studied with reference to possibly nonregular fractions of general factorials. The results are followed by a numerical study and the findings are compared with those based on other design criteria. In the second part, optimal designs are dentified using Bayesian methods. This work is linked with response surface methodology where the first step is to perform factor screening, followed by response surface exploration using different experiment plans. A Bayesian analysis approach is used that aims to achieve both goals using one experiment design. In addition we use a Bayesian design criterion, based on the priors for the analysis approach. This creates an integrated design and analysis framework. To distinguish between competing models, the HD criterion is used, which is based on the pairwise Hellinger distance between predictive densities. Mixed-level fractional factorial designs are commonly used in practice but its aliasing relations have not been studied in full rigor. These designs take the form of a product array. Aliasing patterns of mixed level factorial designs are discussed in the third part. In the fourth part, design of experiment ideas are used to introduce a new global optimization technique called SELC (Sequential Elimination of Level Combinations), which is motivated by genetic algorithms but finds the optimum faster. The two key features of the SELC algorithm, namely, forbidden array and weighted mutation, enhance the performance of the search procedure. Illustration is given with the optimization of three functions, one of which is from Shekel's family. A real example on compound optimization is also given.