Back to search

Michigan State University

Re-conceptualization of modified Angoff standard setting : unified statistical, measurement, cognitive, and social psychological theories

Abstract

dc:description

ABSTRACTRE-CONCEPTUALIZATION OF MODIFIED ANGOFF STANDARD SETTING: UNIFIED STATISTICAL, MEASUREMENT, COGNITIVE, AND SOCIAL PSYCHOLOGICAL THEORIESByIfeoma Chika IyiokeThis dissertation describes a design for training, in accordance with probability judgment heuristics principles, for the Angoff standard setting method. The new training with instruction, practice, and feedback tailored to the probability judgment heuristics principles was called the Heuristic training and the prevailing Angoff method training was called the Normative training. To evaluate effectiveness of the Heuristic training over the Normative training, the researcher ran two empirical studies for this dissertation. The design of the empirical study was a two-way mixed factorial effect ANOVA (2 training methods by 3 rounds of judgment). The empirical studies recommended cut score for the fourth grade mathematics, Michigan Educational Assessment Program (MEAP), Proficient performance category. There were 10 and 12 participants in the Heuristic training and the Normative training, respectively. The participants of the studies were comprised mostly of Michigan State University (MSU) pre-service teachers and teachers in the mid-Michigan area. Two tests were used for the judgments, one for the practice round of judgment called the Practice test and the other for the feedback rounds of judgment called the Real test. Both the Practice and the Real test were comprised of subsets of released Michigan Educational Assessment Program (MEAP), 2005-2006 fourth grade multiple choice items. There were three rounds of probability judgments in both studies namely: the practice and the two feedback rounds. For both training methods, the practice round encompassed pre-requisite tasks for probability judgment and ensued instruction while the feedback rounds followed feedback to the participants on their judgments in the preceding round. The Heuristic and the Normative training methods were evaluated for substantive meaningfulness of the probability judgments and their cut scores derivatives, in relation to the heuristic model assumptions. The methods were also compared for the effectiveness of training interventions of instruction and practice versus feedback.In the practice round of judgment, the training groups performed comparably, the participant's judgments fit the probability judgment heuristic principles, and cut scores were quite reasonable for both groups. Conversely, in the feedback rounds, the participant's judgments deviated from the probability judgment heuristic principles, were considerably less substantively meaningful for the Normative training, and cut scores were positively biased for both groups. The conclusion based on overall findings were that the Heuristic training was more effective than the Normative training and that regardless of training method that instruction and practice activities were more effective than feedback. Intellectual merits of the dissertation, recommendations, study limitations, and directions for future Heuristic training Angoff studies are discussed.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Iyioke, Ifeoma Chika
Contributors dc:contributor
  • Reckase, Mark D.
  • Roeber, Edward D.
  • McAuley, Devin J.
  • Maiti, Tapabrata

Subjects

dc:subject × 4

Rights

dc:rights
Statement dc:rights
  • In Copyright
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
etd:317
isbn:9781303273513
isbn:1303273519
oclc:931742593
umi:3588906
local:Iyioke_grad.msu_0128D_12159

Chain of custody

source
Harvested from
Michigan State University
Base URL
d.lib.msu.edu/oai
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Iyioke, Ifeoma Chika. Re-conceptualization of modified Angoff standard setting : unified statistical, measurement, cognitive, and social psychological theories. 2013. https://doi.org/doi:10.25335/hrce-8m36