The purpose of this study was to investigate the performance of Beaton~{!/~}s MR and MSR fit indices for assessing goodness-of-fit of IRT models. These statistics are based on a standardized residual calculated from an expected and observed response. The investigation was conducted using a Monte Carlo simulation study that varied conditions relevant to testing applications.
This research had three objectives: 1) To identify the sampling distribution of the fit statistics; 2) To assess the Type I error rates under different combinations of manipulated factors; and 3) To investigate the empirical power under different combinations of manipulated factors by introducing different types of model misfit.
The sampling distribution of Beaton~{!/~}s MR and MSR statistics belonged to the family of normal distribution. However, there was no basis for a theoretical normal distribution to test the hypothesis of model-data-fit. Therefore, Monte Carlo resampling methods were required to test the hypothesis of model-data-fit for Beaton~{!/~}s fit statistics.
Using Monte Carlo resampling methods for hypothesis testing, nominal Type I error rates were observed in this study regardless of test length, sample size, Monte Carlo resample size and number of replications. With regard to empirical power, higher power was observed for Beaton~{!/~}s MR statistic than MSR statistic under the condition that H0 was false for the entire test. Under the condition that H0 was false for a subset of test items, higher power for the misfitting item and more false rejections than expected for all the other items were obtained for Beaton~{!/~}s MR statistic. In contrast, reasonable empirical power for the misfitting item and nominal Type I error rates for all the other items were observed for Beaton~{!/~}s MSR statistic. Based on the results of this study, Beaton~{!/~}s MSR fit statistics can be used to assess goodness-of-fit for both shorter (12 items) and longer test (36 items). The recommended sample size is 500 or more, and a Monte Carlo resample size of 100 should be adequate for hypothesis testing.
| Identifer | oai:union.ndltd.org:PITT/oai:PITTETD:etd-12062007-113225 |
| Date | 29 January 2008 |
| Creators | Yin, Yutong |
| Contributors | Clement Stone, Levent Kirisci, Feifei Ye, Suzanne Lane |
| Publisher | University of Pittsburgh |
| Source Sets | University of Pittsburgh |
| Language | English |
| Detected Language | English |
| Type | text |
| Format | application/pdf |
| Source | http://etd.library.pitt.edu/ETD/available/etd-12062007-113225/ |
| Rights | restricted, I hereby certify that, if appropriate, I have obtained and attached hereto a written permission statement from the owner(s) of each third party copyrighted matter to be included in my thesis, dissertation, or project report, allowing distribution as specified below. I certify that the version I submitted is the same as that approved by my advisory committee. I hereby grant to University of Pittsburgh or its agents the non-exclusive license to archive and make accessible, under the conditions specified below, my thesis, dissertation, or project report in whole or in part in all forms of media, now or hereafter known. I retain all other ownership rights to the copyright of the thesis, dissertation or project report. I also retain the right to use in future works (such as articles or books) all or part of this thesis, dissertation, or project report. |
Page generated in 0.0022 seconds