University of British Columbia
Evaluating the methods for block-option selection and the usefulness of latent trait scores in forced-choice tests
Abstract
dc:descriptionMultidimensional forced-choice (MFC) assessments are often used to measure personality, vocational interests, and other non-cognitive attributes. By asking respondents to choose among options, MFC assessments may reduce response biases common in rating scales, such as social desirability, acquiescence, and faking. However, the quality of MFC scores depends on more than the response format. It also depends on how blocks are assembled, how responses are modeled, and how scores are interpreted. This dissertation examines two issues in MFC assessment. Study 1 investigates how option discrimination and option preference parameters affect block option selection. Using simulated data, four block assembly algorithms were compared. Results showed that the algorithm combining both discrimination and preference information produced the most accurate latent trait estimates. Inter-trait correlation was a stronger contextual factor than trait mean distinctness, which primarily moderated differences among the BOS algorithms. Study 2 examines whether latent scores remain useful when the fitted MFC model does not match the decision process that generated the responses. Three models were compared: the Thurstonian item response theory model, the multi-unidimensional pairwise-preference two-parameter model, and the forced-choice diagnostic classification model. Results showed that MFC scores were most useful for within-person trait comparisons, less useful for relative between-person comparisons, and least useful for absolute between-person score interpretation. Model mismatch further reduced score usefulness. Together, the findings show that valid MFC score interpretation requires alignment among block design, model assumptions, trait structure, and intended score use.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy - PhD
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Measurement, Evaluation and Research Methodology
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Wu, Sirui
- Contributors dc:contributor
-
- Wu, Amery D.
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/95743