Return to search

An investigation of the effects of conditioning on two ability estimates in DIF analyses when the data are two-dimensional

Differential Item functioning is present when examinees of the same ability, but belonging to different groups, have differing probabilities of success on an item. Traditionally, DIF detection procedures have been implemented conditioning on total test score. However, if there are group differences on the abilities underlying test performance, and total score is used as the matching criterion, multidimensional item impact may be incorrectly identified as DIF. This study sought to confirm earlier research which demonstrated that multidimensional item impact may be identified as DIF, and then to determine whether conditioning on multiple ability estimates would improve item classification accuracy. Data were generated to simulate responses for 1000 reference group members and 1000 focal group members to two-dimensional tests. The focal group mean on the second ability was one standard deviation less than the reference group mean. The dimensional structure of the tests, the discrimination of the items, and the correlation between the two abilities were varied. Logistic regression and Mantel-Haenszel DIF analyses were conducted using total score as the matching criterion. As anticipated, substantial numbers of items were identified as DIF. Items were then selected into subtests based on item measurement direction. The logistic regression procedure was re-implemented, with subtest scores substituted for total score. In the majority of the conditions simulated, this change in criterion resulted in substantial reductions in Type I errors. The magnitude of the reductions were related to the dimensional structure of the test, and the discrimination of the items. Finally, DIF analyses of two real data sets were conducted, using the same procedures. For one of the two tests, substituting subtest scores for total score resulted in a reduction in number of items identified as DIF. These results suggest that multidimensionality in a data set may have a significant impact on the results of DIF analyses. If total score is used as the matching criterion very high Type I error rates may be expected under some conditions. By conditioning on subtest scores in lieu of total score in logistic regression analyses it may be possible to substantially reduce the number of Type I errors, at least in some circumstances.

Identiferoai:union.ndltd.org:UMASS/oai:scholarworks.umass.edu:dissertations-8783
Date01 January 1993
CreatorsMazor, Kathleen Michele
PublisherScholarWorks@UMass Amherst
Source SetsUniversity of Massachusetts, Amherst
LanguageEnglish
Detected LanguageEnglish
Typetext
SourceDoctoral Dissertations Available from Proquest

Page generated in 0.0838 seconds