Back to search

University of South Florida

Using Stratified Item Selection to Reduce the Number of Items Rated in Standard Setting

Abstract

dc:description

The primary purpose of this study was to evaluate the effectiveness of stratified item sampling in order to reduce the number of items needed in Modified Angoff standard setting studies. Representative subsets of items were extracted from a total of 30 full-length tests based upon content weights, item difficulty, and item discrimination. Cut scores obtained from various size subsets of each test were compared to the full-length test cut score as a measure of generalizability. Applied sampling results indicated that 50% of the full-length test is sufficient to obtain cut scores within one standard error of estimate (SEE) of the full-length test standard, and 70% of the full-length test is sufficient to obtain standards within one percentage point of the full-length test standard. A theoretical sampling procedure indicated that 35% of the full-length test is required to reliably obtain a standard within one SEE of the full-length standard, and 65% of the full-length test is required to fall within one percentage point. The effects of test length, panelist group size, and interrater reliability on the feasibility of stratified item sampling were also examined. However, these standard setting characteristics did not serve as significant predictors of subset generalizability in this study.

Degree

thesis:*
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Psychology
Grantor
University of South Florida
Year dc:date
2011

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Smith, Tiffany Nicole
Contributors dc:contributor
  • Walter Borman, Ph.D.
  • Robert Dedrick, Ph.D.
  • Stephen Stark, Ph.D.
  • Michael Brannick, Ph.D.
  • Douglas Rohrer, Ph.D.

Subjects

dc:subject × 9

Rights

Language dc:language
EN-US

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/3355

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Smith, Tiffany Nicole. Using Stratified Item Selection to Reduce the Number of Items Rated in Standard Setting. Dissertation thesis, University of South Florida, 2011. https://digitalcommons.usf.edu/etd/3355