In this thesis, we first show that the performance of ranking and selection (R&S) procedures in steady-state simulations depends highly on the quality of the variance estimates that are used. We study the performance of R&S procedures using three variance estimators --- overlapping area, overlapping Cramer--von Mises, and overlapping modified jackknifed Durbin--Watson estimators --- that show better long-run performance than other estimators previously used in conjunction with R&S procedures for steady-state simulations. We devote additional study to the development of the new overlapping modified jackknifed Durbin--Watson estimator and demonstrate some of its useful properties. Next, we consider the problem of finding the best simulated system under a primary performance measure, while also satisfying stochastic constraints on secondary performance measures, known as constrained ranking and selection. We first present a new framework that allows certain systems to become dormant, halting sampling for those systems as the procedure continues. We also develop general procedures for constrained R&S that guarantee a nominal probability of correct selection, under any number of constraints and correlation across systems. In addition, we address new topics critical to efficiency of the these procedures, namely the allocation of error between feasibility check and selection, the use of common random numbers, and the cost of switching between simulated systems.

published_or_final_version / abstract / toc / Statistics and Actuarial Science / Master / Master of Philosophy

Thesis (Ph. D.)--University of Missouri-Columbia, 2004. / Typescript. Vita. Includes bibliographical references (leaves 159-165). Also available on the Internet.

Thesis (Ph. D.)--University of Missouri-Columbia, 2004. / Typescript. Vita. Includes bibliographical references (leaves 159-165). Also available on the Internet.

Project report (M.S.) -- Worcester Polytechnic Institute. / Keywords: Bradley-Terry; paired comparison; bayesian statistics. Includes bibliographical references (p. 49).

Thesis (Ph.D.) - University of Queensland, 2006. / Includes bibliography.

Thesis (M.A.)--University of North Carolina at Chapel Hill, 2003. / Includes bibliographical references (leaves 113-114).

Thesis (Ph. D.)--Ohio State University, 2007. / Title from first page of PDF file. Includes bibliographical references (p. 121-124).

This thesis presents three novel contributions to the application as well as development of ranking and selection procedures. Ranking and selection is an important topic in the discrete event simulation literature concerned with the use of statistical approaches to select the best or set of best systems from a set of simulated alternatives. Ranking and selection is comprised of three different approaches: subset selection, indifference zone selection, and multiple comparisons. The methodology addressed in this thesis focuses primarily on the first two approaches: subset selection and indifference zone selection. Our first contribution regards the application of existing ranking and selection procedures to an important body of literature known as system reliability design. If we are capable of modeling a system via a network of arcs and nodes, then the difficult problem of determining the most reliable network configuration, given a set of design constraints, is an optimization problem that we refer to as the network reliability design problem. In this thesis, we first present a novel solution approach for one type of network reliability design optimization problem where total enumeration of the solution space is feasible and desirable. This approach focuses on improving the efficiency of the evaluation of system reliabilities as well as quantifying the probability of correctly selecting the true best design based on the estimation of the expected system reliabilities through the use of ranking and selection procedures, both of which are novel ideas in the system reliability design literature. Altogether, this method eliminates the guess work that was previously associated with this design problem and maintains significant runtime improvements over the existing methodology. Our second contribution regards the development of a new optimization framework for the network reliability design problem that is applicable to any topological and terminal configuration as well as solution sets of any sizes. This framework focuses on improving the efficiency of the evaluation and comparison of system reliabilities, while providing a more robust performance and user-friendly procedure in terms of the input parameter level selection. This is accomplished through the introduction of two novel statistical sampling procedures based on the concepts of ranking and selection: Sequential Selection of the Best Subset and Duplicate Generation. Altogether, this framework achieves the same convergence and solution quality as the baseline cross-entropy approach, but achieves runtime and sample size improvements on the order of 450% to 1500% over the example networks tested. Our final contribution regards the development and extension of the general ranking and selection literature with novel procedures for the problem concerned with the selection of the -best systems, where system means and variances are unknown and potentially unequal. We present three new ranking and selection procedures: a subset selection procedure, an indifference zone selection procedure, and a combined two stage subset selection and indifference zone selection procedure. All procedures are backed by proofs of the theoretical guarantees as well as empirical results on the probability of correct selection. We also investigate the effect of various parameters on each procedure's overall performance.

A method is presented for analysing a 2 x 2 factorial experiment in which the data consist cf relative rankings in pairwise comparisons. Maximum likelihood estimates are developed for the ratings of the various levels of each factor und for the treatment combinations. Likelihood ratio tests of the most important hypotheses likely to arise are derived in detail. The large sample approximations are used. In addition, the method is presented in a manner such that tests of other hypotheses in which the experimenter might be interested can easily be derived. The equations for the analysis of a factorial design of arbitrary size are presented, It can be seen, however, that the complexity of these equations render an attempt at their solution impractical in most cases and more work must be done if a useful method of analysing experiments of this, type is to be found. / Master of Science