Return to search

Estimating attribute-based reliability in cognitive diagnostic assessment

Cognitive diagnostic assessment (CDA) is a testing format that employs a
cognitive model to, first, develop or identify items measuring specific knowledge
and skills and, then, use this model to direct psychometric analyses of
examinees item response patterns to promote diagnostic inferences. The
attribute hierarchy method (AHM, Leighton, Gierl, & Hunka, 2004) is a
psychometric procedure for classifying examinees test item responses into a set
of structured attribute patterns associated with different components from a
cognitive model of task performance. Attribute reliability is a fundamental
concept in cognitive diagnostic assessment because it refers to the consistency
of the decisions made in diagnostic test about examinees mastery of specific
attributes. In this study, an adapted attribute-based reliability estimate was
evaluated in comparison of the standard Cronbachs alpha using simulated data.
Factors expected to influence attribute reliability estimates, including test length,
sample size, model structure, and model-data fit level, were also studied.
Results of this study revealed that the performances of the two attribute-based
reliability estimation indices are comparable; however, the adapted index is
conceptually more meaningful. Test length, model structure, and model-data fit
were shown to impact attribute reliability estimates differentially. Implications
to researchers and practitioners were given based on the simulation results.
Limitations of the present study and future directions were also discussed. / Measurement, Evaluation, and Cognition

Identiferoai:union.ndltd.org:LACETR/oai:collectionscanada.gc.ca:AEU.10048/1052
Date06 1900
CreatorsZhou, Jiawen
ContributorsGierl, Mark J. (Educational Psychology), Leighton, Jacqueline P. (Educational Psychology), Cui, Ying (Educational Psychology), Carbonaro, Michael (Educational Psychology), Gokiert, Rebecca (Faculty of Extension), Jang, Eunice E. (Education, University of Toronto)
Source SetsLibrary and Archives Canada ETDs Repository / Centre d'archives des thèses électroniques de Bibliothèque et Archives Canada
LanguageEnglish
Detected LanguageEnglish
TypeThesis
Format804994 bytes, application/pdf

Page generated in 0.0018 seconds