Cross validation (CV) is widely used for model assessment and comparison. In this thesis, we first review and compare three v-fold CV strategies: best single CV, repeated and averaged CV and double CV. The mean squared errors of the CV strategies in estimating the best predictive performance are illustrated by using simulated and real data examples. The results show that repeated and averaged CV is a good strategy and outperforms the other two CV strategies for finite samples in terms of the mean squared error in estimating prediction accuracy and the probability of choosing an optimal model. In practice, when we need to compare many models, conducting repeated and averaged CV strategy is not computational feasible. We develop an efficient sequential methodology for model comparison based on CV. It also takes into account the randomness in CV. The number of models is reduced via an adaptive, multiplicity-adjusted sequential algorithm, where poor performers are quickly eliminated. By exploiting matching of individual observations, it is sometimes even possible to establish the statistically significant inferiority of some models with just one execution of CV. This adaptive and computationally efficient methodology is demonstrated on a large cheminformatics data set from PubChem. Cross validated mean squared error (CVMSE) is widely used to estimate the prediction mean squared error (MSE) of statistical methods. For linear models, we show how CVMSE depends on the number of folds, v, used in cross validation, the number of observations, and the number of model parameters. We establish that the bias of CVMSE in estimating the true MSE decreases with v and increases with model complexity. In particular, the bias may be very substantial for models with many parameters relative to the number of observations, even if v is large. These results are used to correct CVMSE for its bias. We compare our proposed bias correction with that of Burman (1989), through simulated and real examples. We also illustrate that our method of correcting for the bias of CVMSE may change the results of model selection.

Thesis (Ph. D.)--Technische Universiteit Delft, 2006. / Includes bibliographical references (p. 121-124).

Diss. (sammanfattning) Stockholm : Karol. inst.

Thesis (S.T.M.)--Concordia Theological Seminary, 2006. / Includes bibliographical references (leaves 97-101).

Thesis (Ph.D.)--Duquesne University, 2005. / Title from document title page. Abstract included in electronic submission form. Includes bibliographical references (p. 78-86) and index.

Thesis (S.T.M.)--Concordia Theological Seminary, 2006. / Includes bibliographical references (leaves 97-101).

Thesis (Ph.D.)--York University, 2007. Graduate Programme in History. / Typescript. Includes bibliographical references (leaves 396-421). Also available on the Internet. MODE OF ACCESS via web browser by entering the following URL: http://gateway.proquest.com/openurl?url_ver=Z39.88-2004&res_dat=xri:pqdiss&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&rft_dat=xri:pqdiss:NR32048

Thesis--Copenhagen. / "Literaturangivelser": p. [321]-322.

Thesis (Ph. D.)--Case Western Reserve University, 2008. / Frances Payne Bolton [School of Nursing]. Includes bibliographical references.

Thesis--Catholic University of America. / Bibliography: p. 191-201.