Back to search

North Texas State University

The Characteristics and Properties of the Threshold and Squared-Error Criterion-Referenced Agreement Indices

Abstract

dc:description

Educators who use criterion-referenced measurement to ascertain the current level of performance of an examinee in order that the examinee may be classified as either a master or a nonmaster need to know the accuracy and consistency of their decisions regarding assignment of mastery states. This study examined the sampling distribution characteristics of two reliability indices that use the squared-error agreement function: Livingston's k^2(X,Tx) and Brennan and Kane's M(C). The sampling distribution characteristics of five indices that use the threshold agreement function were also examined: Subkoviak's Pc. Huynh's p and k. and Swaminathan's p and k. These seven methods of calculating reliability were also compared under varying conditions of sample size, test length, and criterion or cutoff score. Computer-generated data provided randomly parallel test forms for N = 2000 cases. From this, 1000 samples were drawn, with replacement, and each of the seven reliability indices was calculated. Descriptive statistics were collected for each sample set and examined for distribution characteristics. In addition, the mean value for each index was compared to the population parameter value of consistent mastery/nonmastery classifications. The results indicated that the sampling distribution characteristics of all seven reliability indices approach normal characteristics with increased sample size. The results also indicated that Huynh's p was the most accurate estimate of the population parameter, with the smallest degree of negative bias. Swaminathan's p was the next best estimate of the population parameter, but it has the disadvantage of requiring two test administrations, while Huynh's p index only requires one administration.

Degree

thesis:*
Grantor dc:publisher
North Texas State University
Year dc:date
1988

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Dutschke, Cynthia F. (Cynthia Fleming)
Contributors dc:contributor
  • Brookshire, William K.
  • Smith, Howard Wellington
  • Thomas, L. Fred (Lawrence Fred)
  • Moore, Alan D.
  • Pavur, Robert J.

Subjects

dc:subject × 5

Rights

dc:rights
Statement dc:rights
  • Public
  • Dutschke, Cynthia F. (Cynthia Fleming)
  • Copyright
  • Copyright is held by the author, unless otherwise noted. All rights reserved.
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
local-cont-no: 1002715062-Dutschke
call-no: 379 N81d no.2818
untcat: b1432221
ark: ark:/67531/metadc331415

Chain of custody

source
Harvested from
University of North Texas
Base URL
digital.library.unt.edu/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Dutschke, Cynthia F. (Cynthia Fleming). The Characteristics and Properties of the Threshold and Squared-Error Criterion-Referenced Agreement Indices. North Texas State University, 1988. https://digital.library.unt.edu/ark:/67531/metadc331415/