The current study examined the impact of unbalanced sample sizes between focal and reference groups on the Type I error rates and DIF
detection rates (power) of five DIF procedures (MH, LR, general IRTLR, IRTLR-b, and SIBTEST). Five simulation factors were used in this
study. Four factors were for generating simulation data and they were sample size, DIF magnitude, group mean ability difference (impact), and
the studied item difficulty. The fifth factor was the DIF method factor that included MH, LR, general IRTLR, IRTLR-b, and SIBTEST. A
repeated-measures ANOVA, where the DIF method factor was the within-subjects variable, was performed to compare the performance of the five
DIF procedures and to discover their interactions with other factors. For each data generation condition, 200 replications were made. Type I
error rates for MH and IRTLR DIF procedures were close to or lower than 5%, the nominal level for different sample size levels. On average,
the Type I error rates for IRTLR-b and SIBTEST were 5.7%, and 6.4%, respectively. In contrast, the LR DIF procedure seems to have a higher
Type I error rate, which ranged from 5.3% to 8.1% with 6.9% on average. When it comes to the rejection rate under DIF conditions, or the DIF
detection rate, the IRTLR-b showed the highest DIF detection rate followed by SIBTEST with averages of 71.8% and 68.4%, respectively.
Overall, the impact of unbalanced sample sizes between reference and focal groups on the performance of DIF detection showed a similar
tendency for all methods, generally increasing DIF detection rates as the total sample size increased. In practice, IRTLR-b, which showed the
best performance for DIF detection rates and controlled for the Type I error rates, should be the choice when the model-data fit is
reasonable. If other non-IRT DIF methods are considered, MH or SIBTEST could be used, depending on which type of error (Type I or II) is more
seriously considered. / A Dissertation submitted to the Department of Educational Psychology and Learning Systems in partial
fulfillment of the requirements for the degree of Doctor of Philosophy. / Fall Semester 2017. / November 6, 2017. / Includes bibliographical references. / Insu Paek, Professor Directing Dissertation; Fred Huffer, University Representative; Betsy Jane Becker,
Committee Member; Yanyun Yang, Committee Member.
| Identifer | oai:union.ndltd.org:fsu.edu/oai:fsu.digital.flvc.org:fsu_604948 |
| Contributors | Alghamdi, Abdullah Ahmed (author), Paek, Insu (professor directing dissertation), Huffer, Fred W. (Fred William) (university representative), Becker, Betsy Jane, 1956- (committee member), Yang, Yanyun (committee member), Florida State University (degree granting institution), College of Education (degree granting college), Department of Educational Psychology and Learning Systems (degree granting departmentdgg) |
| Publisher | Florida State University |
| Source Sets | Florida State University |
| Language | English, English |
| Detected Language | English |
| Type | Text, text, doctoral thesis |
| Format | 1 online resource (85 pages), computer, application/pdf |
Page generated in 0.0022 seconds