Back to search

University of British Columbia

Using cross-classified mixed effects model for validation studies : a flexible and pragmatic validation method

Abstract

dc:description

Validity is the paramount consideration while developing and evaluating tests. To boost validation efforts, this dissertation provides a flexible and pragmatic validation method based on a validity concept with a focus on test score interpretation (Cizek, 2020). In more detail, validation is viewed as an exercise of identifying and explaining desired and undesired effects via a cross-classified mixed effects model (CCMEM). To investigate its potential usefulness, this proposed validation method is applied to two illustrative studies. The first study showcases how to assess reliability and different sources of validity evidence for the ePIRLS 2016 Reading Assessment by utilizing CCMEM. The second study illustrates how to identify potential rating biases and investigate their sources, as well as highlights the importance of a multi-segment (repeated) rating design for competency-based assessment using the Mathematical Quality of Instruction (MQI) instrument. This dissertation expands the utility of CCMEM as a validation tool, and the two application studies showcase the practical versatility of this validation method.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy - PhD
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Measurement, Evaluation and Research Methodology
Grantor dc:publisher
University of British Columbia
Year dc:date
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ji, Xuejun (Ryan)
Contributors dc:contributor
  • Wu, Amery D.

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/86429

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Ji, Xuejun (Ryan). Using cross-classified mixed effects model for validation studies : a flexible and pragmatic validation method. doctoral thesis, University of British Columbia, 2023. http://hdl.handle.net/2429/86429