Back to search

University of British Columbia

Comparing Likert-type and forced-choice formats for assessing preference : a validation study on the College Major Preference Assessment (CMPA)

Abstract

dc:description

This study validated the College Major Preference Assessment (CMPA) by evaluating its effectiveness of using a combination of Likert-type rating scale and forced-choice formats. Following the argument-based approach to validation by Kane (1992), I articulated the interpretation use argument for CMPA and its corresponding warrants and assumptions. The argument was evaluated by the three research questions based on datasets for Education and Psychology: 1) Did the respondents display response set due to Likert-type format of self-assessment? 2) Can the Likert rounds effectively screen out individuals’ “not preferred” majors? 3) Can the forced-choice rounds effectively identify individuals’ favorite major? A two-dimensional latent trait model was tested using structural equation modeling. The results, by and large, confirmed my hypotheses. Both Likert and forced-choice assessments had a very high level of discrimination power for assessing the major trait Favorite (individuals’ inclination towards a major being their favorite). The Likert rounds were relatively less difficult than the forced-choice rounds. The Likert rounds were more reliable for individuals with a relatively lower level of Favorite, and the forced-choice rounds were more reliable for a relatively higher level of Favorite. Moreover, respondents exhibited a response set, shown by the secondary latent trait, which reflected individuals’ perception about the majors when answering Likert-type items. Nonetheless, the assessment of Favorite was not contaminated by perception. All the findings matched the assumptions and warrants for the argument, showing that CMPA worked in the way it was designed for achieving a high level of effectiveness.

Degree

thesis:*
Name thesis:degree_name
Master of Arts - MA
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Measurement, Evaluation and Research Methodology
Grantor dc:publisher
University of British Columbia
Year dc:date
2021

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Wu, Sirui

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/78023

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Wu, Sirui. Comparing Likert-type and forced-choice formats for assessing preference : a validation study on the College Major Preference Assessment (CMPA). master's thesis, University of British Columbia, 2021. http://hdl.handle.net/2429/78023