Michigan State University
Designing p-Optimal Item Pools for Multidimensional Computerized Adaptive Testing
Abstract
dc:descriptionThe interest in multidimensional computerized adaptive testing (MCAT) has grown considerably over the last few years. While a significant amount of research has been conducted on item selection and ability estimation methods for MCAT, few studies specifically addressed the item pool design for MCAT. To ensure a proper functioning of MCAT, a well-designed item pool is imperative. A well-designed item pool should consist of a number of well-balanced items that achieve appropriate test precision, item usage, as well as lower the cost of item creation. One method to develop such an item pool is the p-optimality method, which is proposed by Reckase (2003 & 2007) for unidimensional CAT. This paper aims to develop p-optimal item pools for MCAT by extending the Reckase's method to a multidimensional context. The extension includes the generation of a multidimensional optimal item based on the D-Optimality item selection creation, the definition of the MDIFF-bin to describe multidimensional item succinctly for item pool design, and the interpretation for the p-optimal item pool in a multidimensional context. In this paper, a total of 24 p-optimal item pools were designed and then developed for different test specification, with different correlation among dimensions, based on different bin size, and under the condition with or without item exposure control. The characteristics for the 24 p-optimal item pools are summarized. A simulation study was conducted to evaluate the performance of the p-optimal item pools against baseline pools existing in research literature. Results show that p-optimal item pools achieve similar levels of measurement accuracy as baseline pools, but they consist of fewer items and perform better in terms of item pool usage and test security. The characteristics and the performance of the p-optimal item pools are affected by factors such as test specification, correlation among dimensions, bin size, and item exposure control. The results in this study can provide a general guideline for the item pool development for MCAT. More importantly, because the p-optimal item pool is specifically tailored to the MCAT programs, the p-optimal item pool design procedure described in this study can be adapted to other MCAT programs with different features and purposes. The end product of the p-optimal item pool design can be used as an instructive guide for item creation, item pool development, and item pool management.
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Mao, Liyang
- Contributors dc:contributor
-
- Reckase, Mark
- Roeber, Edward
- Martineau, Joseph
- Houang, Richard
Subjects
dc:subject × 2Rights
dc:rights- Statement dc:rights
-
- In Copyright
- Language dc:language
- English
Identifiers
dc:identifier.*- Identifier
-
etd:3301
isbn:9781321165074
isbn:1321165072
oclc:973326153
umi:3635440
local:Mao_grad.msu_0128D_13193 - OAI record oai:identifier
- https://d.lib.msu.edu/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Ad.lib.msu.edu%3Aetd_3301