University of South Florida
The Performance of the Linear Logistic Test Model When the Q-Matrix is Misspecified: A Simulation Study
Abstract
dc:descriptionA simulation study was conducted to explore the performance of the linear logistic test model (LLTM) when the relationships between items and cognitive components were misspecified. Factors manipulated included percent of misspecification (0%, 1%, 5%, 10%, and 15%), form of misspecification (under-specification, balanced misspecification, and over-specification), sample size (20, 40, 80, 160, 320, 640, and 1280), Q-matrix density (60% and 46%), number of items (20, 40, and 60 items), and skewness of person ability distribution (-0.5, 0, and 0.5). Statistical bias, root mean squared error, confidence interval coverage, confidence interval width, and pairwise cognitive components correlations were computed. The impact of the design factors were interpreted for cognitive components, item difficulty, and person ability parameter estimates. The simulation provided rich results and selected key conclusions include (a) SAS works superbly when estimating LLTM using a marginal maximum likelihood approach for cognitive components and an empirical Bayes estimation for person ability, (b) parameter estimates are sensitive to misspecification, (c) under-specification is preferred to over-specification of the Q-matrix, (d) when properly specified the cognitive components parameter estimates often have tolerable amounts of root mean squared error when the sample size is greater than 80, (e) LLTM is robust to the density of Q-matrix specification, (f) the LLTM works well when the number of items is 40 or greater, and (g) LLTM is robust to a slight skewness of the person ability distribution. In sum, the LLTM is capable of identifying conceptual knowledge when the Q-matrix is properly specified, which is a rich area for applied empirical research.
Degree
thesis:*- Level thesis:degree_level
- Dissertation
- Discipline thesis:degree_discipline
- Educational Measurement and Research
- Grantor
- University of South Florida
- Year dc:date
- 2013
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Macdonald, George T.
- Contributors dc:contributor
-
- Jeffrey D. Kromrey, Ph.D.
- Yi-Hsin Chen, Ph.D.
- Robert F. Dedrick, Ph.D.
- Gladis Kersaint, Ph.D.
Subjects
dc:subject × 5Rights
- Language dc:language
- EN-US
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://digitalcommons.usf.edu/etd/5065