abstract: Recent legislation allowing educational agencies to use Response to Intervention (RTI) in determining whether a child has a specific learning disability, coupled with a focus on large-scale testing and accountability resulted in the increasing use of curriculum based measurement (CBM) as a tool for understanding students' progress towards state standards, particularly in reading through the use of oral reading fluency measures. Extensive evidence of oral reading fluency's predictability of reading comprehension exists, but little research on differential effects across racial, gender, and socioeconomic subgroups is available. This study investigated racial, gender, and socioeconomic bias in DIBELS Oral Reading Fluency (DIBELS ORF) probes predictive and concurrent relationship with MAP reading comprehension scores for African American and Caucasian students. Participants were 834 second through fifth grade students in a school district located in a southeastern US state. The dataset consisted of student fall and spring DIBELS ORF scores and spring MAP reading comprehension scores. Concurrent correlation results between spring DIBELS ORF and MAP reading comprehension scores were moderate to large and statistically significant across all grades and demographic groups; however, correlations between fall DIBELS ORF and MAP reading comprehension scores were generally weak. Stepwise multiple regression analyses were used to examine the best variable, or combination of variables, in predicting MAP reading comprehension scores. Models differed for each grade level; however, spring DIBELS ORF scores were always included, whether alone or in combination with demographic variables, in the best prediction model. Potthoff's procedure was used to simultaneously test for slope and intercept differences among regression equations to determine if DIBELS ORF scores from fall and spring differentially predicted MAP reading comprehension scores across demographic groups. Nine of 24 simultaneous contrasts demonstrated a significant effect; seven were related to race, one was related to gender, and one was related to socioeconomic status. Racial bias in predicting MAP reading comprehension performance from spring DIBELS ORF was found. Differential prediction among gender and SES groups was not consistent indicating little to no practical significance. Results are discussed in the context of practical implications of differential validity, both predictive and concurrent, and potential impact on disproportionality. / Dissertation/Thesis / Ph.D. Educational Psychology 2013

The purpose of this research was to investigate the relation between measurement bias at the item level (differential item functioning, dif) and predictive bias at the test score level. Dif was defined as a difference in the probability of getting a test item correct for examinees with the same ability but from different subgroups. Predictive bias was defined as a difference in subgroup regression intercepts and/or slopes in predicting a criterion. Data were simulated by computer. Two hypothetical subgroups (a reference group and a focal group) were used. The predictor was a composite score on a dimensionally complex test with 60 items. Sample size (35, 70, and 105 per group), validity coefficient (.3 or .5), and the mean difference on the predictor (0, .33, .66, and 1 standard deviation, sd) and the criterion (0 and .35 sd) were manipulated. The percentage of items showing dif (0%, 15%, and 30%) and the effect size of dif (small = .3, medium = .6, and large = .9) were also manipulated. Each of the 432 conditions in the 3 x 2 x 4 x 2 x 3 x 3 design was replicated 500 times. For each replication, a predictive bias analysis was conducted, and the detection of predictive bias against each subgroup was the dependent variable. The percentage of dif and the effect size of dif were hypothesized to influence the detection of predictive bias; hypotheses were also advanced about the influence of sample size and mean subgroup differences on the predictor and criterion. Results indicated that dif was not related to the probability of detecting predictive bias against any subgroup. Results were inconsistent with the notion that measurement bias and predictive bias are mutually supportive, i.e., the presence (or absence) of one type of bias is evidence in support of the presence (or absence) of the other type of bias. Sample size and mean differences on the predictor/criterion had direct and indirect effects on the probability of detecting predictive bias against both reference and focal groups. Implications for future research are discussed.