The purpose of this investigation was to determine the influence of item response theory and different types of judges on a standard. The iterative Angoff standard setting method was employed by all judges to determine a cut-off score for a public school district-wide criterion-reformed test. The analysis of variance of the effect of judge type and standard setting method on the central tendency of the standard revealed the existence of an ordinal interaction between judge type and method. Without any knowledge of p-values, one judge group set an unrealistic standard. A significant disordinal interaction was found concerning the effect of judge type and standard setting method on the variance of the standard. A positive covariance was detected between judges' minimum pass level estimates and empirical item information. With both p-values and b-values, judge groups had mean minimum pass levels that were positively correlated (ranging from .77 to .86), regardless of the type of information given to the judges. No differences in correlations were detected between different judge types or different methods. The generalizability coefficients and phi indices for 12 judges included in any method or judge type were acceptable (ranging from .77 to .99). The generalizability coefficient and phi index for all 24 judges were quite high (.99 and .96, respectively).
| Identifer | oai:union.ndltd.org:unt.edu/info:ark/67531/metadc332675 |
| Date | 08 1900 |
| Creators | Hamberlin, Melanie Kidd |
| Contributors | Schumacker, Randall, Anderson, Gary Weldon, McCallon, Earl, Haynes, Jack R. |
| Publisher | University of North Texas |
| Source Sets | University of North Texas |
| Language | English |
| Detected Language | English |
| Type | Thesis or Dissertation |
| Format | vi, 84 leaves : ill., Text |
| Rights | Public, Hamberlin, Melanie Kidd, Copyright, Copyright is held by the author, unless otherwise noted. All rights reserved. |
Page generated in 0.0015 seconds