University of British Columbia
Using cross-classified mixed effects model for validation studies : a flexible and pragmatic validation method
Abstract
dc:descriptionValidity is the paramount consideration while developing and evaluating tests. To boost validation efforts, this dissertation provides a flexible and pragmatic validation method based on a validity concept with a focus on test score interpretation (Cizek, 2020). In more detail, validation is viewed as an exercise of identifying and explaining desired and undesired effects via a cross-classified mixed effects model (CCMEM). To investigate its potential usefulness, this proposed validation method is applied to two illustrative studies. The first study showcases how to assess reliability and different sources of validity evidence for the ePIRLS 2016 Reading Assessment by utilizing CCMEM. The second study illustrates how to identify potential rating biases and investigate their sources, as well as highlights the importance of a multi-segment (repeated) rating design for competency-based assessment using the Mathematical Quality of Instruction (MQI) instrument. This dissertation expands the utility of CCMEM as a validation tool, and the two application studies showcase the practical versatility of this validation method.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy - PhD
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Measurement, Evaluation and Research Methodology
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2023
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Ji, Xuejun (Ryan)
- Contributors dc:contributor
-
- Wu, Amery D.
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/86429