Cognitive diagnostic assessment (CDA) is a testing format that employs a
cognitive model to, first, develop or identify items measuring specific knowledge
and skills and, then, use this model to direct psychometric analyses of
examinees item response patterns to promote diagnostic inferences. The
attribute hierarchy method (AHM, Leighton, Gierl, & Hunka, 2004) is a
psychometric procedure for classifying examinees test item responses into a set
of structured attribute patterns associated with different components from a
cognitive model of task performance. Attribute reliability is a fundamental
concept in cognitive diagnostic assessment because it refers to the consistency
of the decisions made in diagnostic test about examinees mastery of specific
attributes. In this study, an adapted attribute-based reliability estimate was
evaluated in comparison of the standard Cronbachs alpha using simulated data.
Factors expected to influence attribute reliability estimates, including test length,
sample size, model structure, and model-data fit level, were also studied.
Results of this study revealed that the performances of the two attribute-based
reliability estimation indices are comparable; however, the adapted index is
conceptually more meaningful. Test length, model structure, and model-data fit
were shown to impact attribute reliability estimates differentially. Implications
to researchers and practitioners were given based on the simulation results.
Limitations of the present study and future directions were also discussed. / Measurement, Evaluation, and Cognition
| Identifer | oai:union.ndltd.org:LACETR/oai:collectionscanada.gc.ca:AEU.10048/1052 |
| Date | 06 1900 |
| Creators | Zhou, Jiawen |
| Contributors | Gierl, Mark J. (Educational Psychology), Leighton, Jacqueline P. (Educational Psychology), Cui, Ying (Educational Psychology), Carbonaro, Michael (Educational Psychology), Gokiert, Rebecca (Faculty of Extension), Jang, Eunice E. (Education, University of Toronto) |
| Source Sets | Library and Archives Canada ETDs Repository / Centre d'archives des thèses électroniques de Bibliothèque et Archives Canada |
| Language | English |
| Detected Language | English |
| Type | Thesis |
| Format | 804994 bytes, application/pdf |
Page generated in 0.0021 seconds