Diss. (sammanfattning) Stockholm : Karol. inst. / Härtill 5 uppsatser.

Diss. (sammanfattning) Stockholm : Karol. inst. / Härtill 5 uppsatser.

Thesis (Ph. D.)--University of Wyoming, 2006. / Title from PDF title page (viewed on June 30, 2008). Includes bibliographical references (p. 130-139).

Thesis (Ph. D.)--Ohio State University, 2005. / Title from first page of PDF file. Document formatted into pages; contains xvi, 276 p.; also includes graphics. Includes bibliographical references (p. 262-276). Available online via OhioLINK's ETD Center.

Thesis (Ph. D.)--University of Florida, 1984. / Description based on print version record. Typescript. Vita. Includes bibliographical references (leaves 195-208).

Thesis (Ph. D.)--Bryn Mawr college, 1923. / Vita. Bibliography: p. 131-141.

Thesis (M.S.)--University of North Carolina at Wilmington, 2006. / Includes bibliographical references (leaves: 51-56)

Thesis (M.A.)--State University of New York at Binghamton, Psychology Dept., 2005. / Includes bibliographical references.

The purpose of this study was to compare linear fixed length test (LFT), multi stage test (MST), and computer adaptive test (CAT) designs under three levels of item pool quality, two levels of match between test and item pool content specifications, two levels of test length, and several levels of exposure control expected to be practical for a number of testing programs. This design resulted in 132 conditions that were evaluated using a simulation study with 9000 examinees on several measures of overall measurement precision including reliability, the mean error and root mean squared error between true and estimated ability levels, classification precision including decision accuracy, false positive and false negative rates, and Kappa for cut scores corresponding to 30%, 50%, and 85% failure rates, and conditional measurement precision with the conditional root mean squared error between true and estimated ability levels conditioned on 25 true ability levels. Test reliability, overall and conditional measurement precision, and classification precision increased with item pool quality and test length, and decreased with less adequate match between item pool and test specification match. In addition, as the maximum exposure rate decreased and the type of exposure control implemented became more restrictive, test reliability, overall and conditional measurement precision, and classification precision decreased. Within item pool quality, match between test and item pool content specifications, test length, and exposure control, CAT designs showed superior psychometric properties as compared to MST designs which in turn were superior to LFT designs. However, some caution is warranted in interpreting these results since the ability of the automated test assembly software to construct test that met specifications was limited in conditions where pool usage was high. The practical importance of the differences between test designs on the evaluation criteria studied is discussed with respect to the inferences test users seek to make from test scores and nonpsychometric factors that may be important in some testing programs.

The purpose of the study was to investigate the impact of several testing variables—level of item quality, item bank size, placement of passing score, and computer-based test design, on two important indicators of the quality of credentialing exams, decision consistency and decision accuracy. The computer-based test designs of interest in this study were linear parallel form tests, multi-stage tests, and computerized adaptive tests. The choice of test design model plays an important role in both minimizing decision-making error and increasing test efficiency. However, of equal, if not greater, importance are the resources available to test developers: the number of items in the bank, and the quality of the items. Simulation studies were conducted to investigate the effects of these testing variables on the accuracy and consistency of binary decision-making of credentialing exams. These variables are all very essential and manipulable in practice and therefore their roles are especially worthy of investigation. To the extent possible, realistic situations were simulated to increase the generalizability of the findings. The main findings from the study were as follows: (1) improvements in item quality had the desirable effect of increasing decision accuracy and decision consistency by a practically significant amount; (2) doubling bank size helped significantly in lowering item exposure but had little direct impact on decision accuracy and decision consistency; (3) the location of the passing score significantly impacted on test length necessary to achieve desirable levels of test statistics, and (4) all three computer-based test designs produced comparable results in the conditions simulated. Two main conclusions can be drawn from the findings. First, steps to increase item quality such as improving item writer training, cloning the best items, and improving and extending field testing of new items, offer the potential for improving the statistical characteristics of pass-fail credential exam decisions. Second, the general attractiveness of new computer-based test designs may be less important when pass-fail decisions rather than precise ability estimates are the focus of the examination. Clearly, more research to pursue these two conclusions, and confirm them, if possible, is in order.