This study had three purposes: (1) to identify differential item functioning (DIF) on the PPVT-III (Forms A & B) using a partial correlation method, (2) to find a consistent pattern in items identified as underestimating ability in each ethnic minority group, and (3) to compare findings from an expert judgment method and a partial correlation method. Hispanic, African American, and white subjects for the study were provided by American Guidance Service (AGS) from the standardization sample of the PPVT-III; English language learners (ELL) of Mexican descent were recruited from school districts in Central and South Texas. Content raters were all self-selected volunteers, each had advanced degrees, a career in education, and no special expertise of ELL or ethnic minorities. Two groups of teachers participated as judges for this study. The "expert" group was selected because of their special knowledge of ELL students of Mexican descent. The control group was all regular education teachers with limited exposure to ELL. Using the partial correlation method, DIF was detected within each group comparison. In all cases except with the ELL on form A of the PPVT-III, there were no significant differences in numbers of items found to have significant positive correlations versus significant negative correlations. On form A, the ELL group comparison indicated more items with negative correlation than positive correlation [χ2 (1) = 5.538; p=.019]. Among the items flagged as underestimating ability of the ELL group, no consistent trend could be detected. Also, it was found that none of the expert judges could adequately predict those items that would underestimate ability for the ELL group, despite expertise. Discussion includes possible consequences of item placement and recommendations regarding further research and use of the PPVT-III.

In the field of education, decisions are influenced by the results of various high stakes measures. Investigating the presence of differential item functioning (DIF) in a set of items ensures that results from these measures are valid. For example, if an item measuring math self-efficacy is identified as having DIF then this indicates that some other characteristic (e.g. gender) other than the latent trait of interest may be affecting an examinee’s score on that particular item. The use of hierarchical generalized linear modeling (HGLM) enables the modeling of items nested within examinees, with person-level predictors added at level-2 for DIF detection. Unlike traditional DIF detection methods that require a reference and focal group, HGLM allows the modeling of a continuous person-level predictor. This means that instead of dichotomizing a continuous variable associated with DIF into a focal and reference group, the continuous variable can be added at level-2. Further benefits of HGLM are discussed in this study. This study is an extension of work done by Williams and Beretvas (2006) where the use of HGLM with polytomous items (PHGLM) for detection of DIF was illustrated. In the Williams and Beretvas study, the PHGLM was compared with the generalized Mantel-Haenszel (GMH), for DIF detection and it was found that the two performed similarly. A Monte Carlo simulation study was conducted to evaluate HGLM’s power to detect DIF and its associated Type 1 error rates using the constrained form of Muraki’s Rating Scale Model (Muraki, 1990) as the generating model. The two methods were compared when DIF was associated with a continuous variable which was dichotomized for the GMH and used as a continuous person-level predictor with PHGLM. Of additional interest in this study was the comparison of HGLM’s performance with that of the GMH under a variety of DIF and sample size conditions. Results showed that sample size, sample size ratio and DIF magnitude substantially influenced the power performance for both GMH and HGLM. Furthermore, the power performance associated with the GMH was comparable to HGLM for conditions with large sample sizes. The mean performance for both DIF detection methods showed good Type I error control.

ABSTRACT A MONTE CARLO STUDY INVESTIGATING MISSING DATA, DIFFERENTIAL ITEM FUNCTIONING, AND EFFECT SIZE by Phyllis Garrett The use of polytomous items in assessments has increased over the years, and as a result, the validity of these assessments has been a concern. Differential item functioning (DIF) and missing data are two factors that may adversely affect assessment validity. Both factors have been studied separately, but DIF and missing data are likely to occur simultaneously in real assessment situations. This study investigated the Type I error and power of several DIF detection methods and methods of handling missing data for polytomous items generated under the partial credit model. The Type I error and power of the Mantel and ordinal logistic regression were compared using within-person mean substitution and multiple imputation when data were missing completely at random. In addition to assessing the Type I error and power of DIF detection methods and methods of handling missing data, this study also assessed the impact of missing data on the effect size measure associated with the Mantel, the standardized mean difference effect size measure, and ordinal logistic regression, the R-squared effect size measure. Results indicated that the performance of the Mantel and ordinal logistic regression depended on the percent of missing data in the data set, the magnitude of DIF, and the sample size ratio. The Type I error for both DIF detection methods varied based on the missing data method used to impute the missing data. Power to detect DIF increased as DIF magnitude increased, but there was a relative decrease in power as the percent of missing data increased. Additional findings indicated that the percent of missing data, DIF magnitude, and sample size ratio also influenced the effect size measures associated with the Mantel and ordinal logistic regression. The effect size values for both DIF detection methods generally increased as DIF magnitude increased, but as the percent of missing data increased, the effect size values decreased.

In response to public concern over fairness in testing, conducting a differential item functioning (DIF) analysis is now standard practice for many large-scale testing programs (e.g., Scholastic Aptitude Test, intelligence tests, licensing exams). As highlighted by the Standards for Educational and Psychological Testing manual, the legal and ethical need to avoid bias when measuring examinee abilities is essential to fair testing practices (AERA-APA-NCME, 1999). Likewise, the development of statistical and substantive methods of investigating DIF is crucial to the goal of designing fair and valid educational and psychological tests. Douglas, Roussos and Stout (1996) introduced the concept of item bundle DIF and the implications of differential bundle functioning (DBF) for identifying the underlying causes of DIF. Since then, several studies have demonstrated DIF/DBF analyses within the framework of “unintended” multidimensionality (Oshima & Miller, 1992; Russell, 2005). Russell (2005), in particular, examined the effect of secondary traits on DBF/DTF detection. Like Russell, this study created item bundles by including multidimensional items on a simulated test designed in theory to be unidimensional. Simulating reference group members to have a higher mean ability than the focal group on the nuisance secondary dimension, resulted in DIF for each of the multidimensional items, that when examined together produced differential bundle functioning. The purpose of this Monte Carlo simulation study was to assess the Type I error and power performance of SIBTEST (Simultaneous Item Bias Test; Shealy & Stout, 1993a) for DBF analysis under various conditions with simulated data. The variables of interest included sample size and ratios of reference to focal group sample sizes, correlation between primary and secondary dimensions, magnitude of DIF/DBF, and angular item direction. Results showed SIBTEST to be quite powerful in detecting DBF and controlling Type I error for almost all of the simulated conditions. Specifically, power rates were .80 or above for 84% of all conditions and the average Type I error rate was approximately .05. Furthermore, the combined effect of the studied variables on SIBTEST power and Type I error rates provided much needed information to guide further use of SIBTEST for identifying potential sources of differential item/bundle functioning.

WAIS-III (Wechsler intelekto matavimo skalės suaugusiems trečioji versija) – vienas plačiausiai pasaulyje naudojamų intelekto matavimo instrumentų. Vienas iš testo šališkumo šaltinių yra užduočių atlikimo skirtumai atskirose grupėse, lyginant asmenis, turinčius tokius pačius gebėjimus – skirtingas užduočių funkcionavimas. Tyrimo tikslas buvo įvertinti Lietuvoje adaptuojamos WAIS–III verbalinės testo dalies užduočių šališkumą. Užduočių funkcionavimas buvo tikrinamas lyginant tiriamųjų grupes pagal lytį (172 moterys ir 128 vyrai) ir išsilavinimą (209 tiriamiejis su viduriniu ir žemesniu išsilavinimu ir 89 tiriamieji su aukštesniuoju ir aukštuoju išsilavinimu). Užduočių atlikimas buvo analizuojamas iš viso šešiuose WAIS–III subtestuose (Paveikslėlių užbaigimo, Žodyno, Panašumų, Aritmetikos, Informacijos ir Supratingumo). Rasta, kad iš 148 analizuotų užduočių 20 skirtingai funkcionavo vyrų ir moterų grupėse bei 19 grupėse pagal išsilavinimą (daugiausia skirtumų rasta Žodyno ir Informacijos subtestuose). Daugiausia buvo užduoties sunkumo skirtumų, kurių skaičius buvo labai panašus ir lyginant vyrų ir moterų užduočių atlikimą, ir tiriamuosius pagal išsilavinimą. Rezultatai parodė, kad iš užduočių, kurios skyrėsi skiriamąja geba, yra žymiai daugiau, pagal gebėjimus geriau diferencijuojančių moteris nei vyrus ir asmenis su viduriniu ir žemesniu išsilavinimu. / WAIS–III (Wechsler Adult‘s Intelligence Scale - Third edition) – is one of the most widely used intelligence measuring scale in the world, which has also been adapted in Lithuania. A significant part of researches is related to scale‘s bias – one of it‘s sources is Differential item functioning in separate groups, when participants have the same abilities. The aim of this work was to assess the bias of verbal subtests items in Lithuanian WAIS-III version. Differential item functioning was tested comparing groups by sex (128 males and 172 females) and education (209 participants had a secondary and lower education, 89 had further and university education), in six WAIS-III subtests (Picture Completion, Vocabulary, Similarities, Arithmetic, Information and Comprehension). Data analysis showed that 20 items from 148 were functioning differently for males and females, and 19 items functioned differently in groups by education (the differences were mostly in Vocabulary and Information subtests). The biggest part of differences were of uniform DIF – the number was very similar in males and females groups, also in education groups. The bigger part of nonuniform DIF were more discriminating for females than males and participants with secondary and lower education.

国立情報学研究所で電子化したコンテンツを使用している。

国立情報学研究所で電子化したコンテンツを使用している。

国立情報学研究所で電子化したコンテンツを使用している。

Esta dissertação consiste primordialmente na descrição e interpretação do item lexical mas nas crônicas de Luis Fernando Verissimo, seguindo uma abordagem sintático-semântico-discursiva. O vocábulo mas é identificado nos textos de acordo com sua posição nos enunciados e analisado a partir desse ponto de inserção. Devido à variedade de emprego de tal item lexical, relacionando termos, orações, períodos, parágrafos e porções maiores de texto, evidenciou-se seu papel na construção da coesão, da progressão textual, da coerência, da delimitação de etapas discursivas e da orientação argumentativa dos segmentos de discurso. Pretendeu-se uma melhor compreensão das crônicas de Luis Fernando Verissimo através da marca lingüística do item lexical mas. / This dissertation consists of primarily in the description and interpretation of the lexical item but in cronics of Luis Fernando Verissimo following a syntax-semantics-discursive approach. The word "but" is identified in texts in accordance with its position in a sentence and analyzed from this insertion point. Due to the variety of implementations of such lexical item and related terms, sentences, periods, paragraphs and larger portions of text, its role in the construction of cohesion, coherent textual progression, limitations of discursive phases, and the argumentative orientation of the discurses was evidenced. A better understand of the cronics of Luis Fernando Verissimo is accomplished through the linguistic makr of the lexical item but.

Esta dissertação consiste primordialmente na descrição e interpretação do item lexical mas nas crônicas de Luis Fernando Verissimo, seguindo uma abordagem sintático-semântico-discursiva. O vocábulo mas é identificado nos textos de acordo com sua posição nos enunciados e analisado a partir desse ponto de inserção. Devido à variedade de emprego de tal item lexical, relacionando termos, orações, períodos, parágrafos e porções maiores de texto, evidenciou-se seu papel na construção da coesão, da progressão textual, da coerência, da delimitação de etapas discursivas e da orientação argumentativa dos segmentos de discurso. Pretendeu-se uma melhor compreensão das crônicas de Luis Fernando Verissimo através da marca lingüística do item lexical mas. / This dissertation consists of primarily in the description and interpretation of the lexical item but in cronics of Luis Fernando Verissimo following a syntax-semantics-discursive approach. The word "but" is identified in texts in accordance with its position in a sentence and analyzed from this insertion point. Due to the variety of implementations of such lexical item and related terms, sentences, periods, paragraphs and larger portions of text, its role in the construction of cohesion, coherent textual progression, limitations of discursive phases, and the argumentative orientation of the discurses was evidenced. A better understand of the cronics of Luis Fernando Verissimo is accomplished through the linguistic makr of the lexical item but.