The purpose of this study was to determine if using a multistage approach for the empirical selection of anchor items would lead to more accurate DIF detection rates than the anchor selection methods proposed by Kopf, Zeileis, & Strobl (2015b). A simulation study was conducted in which the sample size, percentage of DIF, and balance of DIF were manipulated. The outcomes of interest were true positive rates, false positive rates, familywise false positive rates, anchor contamination rates, and familywise anchor contamination rates. Results showed the proposed multistage methods produced lower anchor contamination rates than the non-multistage methods under some conditions, but there were generally no meaningful differences in true positive and false positive rates.

Differential item functioning (DIF) is a psychometric issue routinely considered in educational and psychological assessment. However, it has not been studied in the context of a recently developed componential statistical model, the model with internal restrictions on item difficulty (MIRID; Butter, De Boeck, & Verhelst, 1998). Because the MIRID requires test questions measuring either single or multiple cognitive processes, it creates a complex environment for which traditional DIF methods may be inappropriate. This dissertation sought to extend the MIRID framework to detect DIF at the item-group level and the individual-item level. Such a model-based approach can increase the interpretability of DIF statistics by focusing on item characteristics as potential sources of DIF. In particular, group-level DIF may reveal comparative group strengths in certain secondary constructs. A simulation study was conducted to examine under different conditions parameter recovery, Type I error rates, and power of the proposed approach. Factors manipulated included sample size, magnitude of DIF, distributional characteristics of the groups, and the MIRID DIF models corresponding to discrete sources of differential functioning. The impact of studying DIF using wrong models was investigated. The results from the recovery study of the MIRID DIF model indicate that the four delta (i.e., non-zero value DIF) parameters were underestimated whereas item locations of the four associated items were overestimated. Bias and RMSE were significantly greater when delta was larger; larger sample size reduced RMSE substantially while the effects from the impact factor were neither strong nor consistent. Hypothesiswise and adjusted experimentwise Type I error rates were controlled in smaller delta conditions but not in larger delta conditions as estimates of zero-value DIF parameters were significantly different from zero. Detection power of the DIF model was weak. Estimates of the delta parameters of the three group-level DIF models, the MIRID differential functioning in components (DFFc), the MIRID differential functioning in item families (DFFm), and the MIRID differential functioning in component weights (DFW), were acceptable in general. They had good hypothesiswise and adjusted experimentwise Type I error control across all conditions and overall achieved excellent detection power. When fitting the proposed models to mismatched data, the false detection rates were mostly beyond the Bradley criterion because the zero-value DIF parameters in the mismatched model were not estimated adequately, especially in larger delta conditions. Recovery of item locations and component weights was also not adequate in larger delta conditions. Estimation of these parameters was more or less affected adversely by the DIF effect simulated in the mismatched data. To study DIF in MIRID data using the model-based approach, therefore, more research is necessary to determine the appropriate procedure or model to implement, especially for item-level differential functioning.

The present study examined the structural construct validity of the LoTi Digital-Age Survey, a measure of teacher instructional practices with technology in the classroom. Teacher responses (N = 2840) from across the United States were used to assess factor structure of the instrument using both exploratory and confirmatory analyses. Parallel analysis suggests retaining a five-factor solution compared to the MAP test that suggests retaining a three-factor solution. Both analyses (EFA and CFA) indicate that changes need to be made to the current factor structure of the survey. The last two factors were composed of items that did not cover or accurately measure the content of the latent trait. Problematic items, such as items with crossloadings, were discussed. Suggestions were provided to improve the factor structure, items, and scale of the survey.

No description available.

The Progress in International Reading Literacy Study (PIRLS) is a large-scale reading comprehension assessment, which assesses Grade 4 learners’ reading literacy achievement. The findings from the last cycle of PIRLS 2016 indicated that South African Grade 4 and 5 learners performed poorly in reading comprehension. This finding confirms the previous cycles’ results where South African learners achieved the lowest results across the participating countries. Approximately eight out of ten Grade 4 learners cannot read for meaning in any of the tested languages. Due to the poor results in PIRLS, the President of South Africa stated that every ten-year old child should be able to read for meaning, thus cementing reading literacy as a national aim. The aim of this mixed methods research was to determine whether the PIRLS Literacy 2016 and PIRLS 2016 limited release texts are equivalent across languages, specifically English, Afrikaans and isiZulu. Four research sub-questions were explored to assist in addressing the main research question posed by this study: To what extent are the PIRLS 2016 released texts in English, Afrikaans and isiZulu, in Grade 4 and Grade 5 equivalent? As this study took the form of a sequential explanatory mixed methods approach, the first phase investigated the South African Grade 4 and 5 results by firstly looking at descriptive statistics, such as percentages and means. After the initial exploration of the data, I conducted Rasch analyses to determine whether the items from the limited release texts showed measurement invariance – in other words, whether the items behaved differently for different groups of learners. As part of the Rasch analyses, individual item-fit statistics and differential item functioning (DIF) were conducted using RUMM2030. In phase two, the limited release texts were analysed by experts who attended workshops and completed open-ended questionnaires regarding the equivalence of the identified texts. The qualitative phase was conducted in order to complement and extend on the quantitative findings of phase one. The findings revealed that the limited release texts, with their accompanying items, were not equivalent across the different languages. However, by looking at the items that displayed DIF, there is not a clear pattern as the items did not universally favour one language nor did the texts discriminate universally against a particular language. An in-depth look at the texts and items themselves revealed that the Flowers on the Roof text is considered the poorest translation into Afrikaans and isiZulu. Overall, all the texts were considered to be appropriate for South African learners as the texts made use of rich vocabulary and introduced the learners to new ideas and concepts. Thus, this study offers new insights into the equivalence of the PIRLS assessments as well as possible reasons for the non-equivalence for each of the limited release texts. Based on the findings of this study, recommendations and further research are provided. / Thesis (PhD)--University of Pretoria, 2020. / Science, Mathematics and Technology Education / PhD / Unrestricted

Multilingualism in the classroom is one of the many challenges found in the cumbersome bag that the South African education system is carrying over its shoulders at present. Globalisation and migration have added to the burden as factors adding further diversity to the already diverse classroom. In South Africa the spotlight is focused on equality. Equality is expected in the education system, and in the classroom and especially in tests. With 11 official languages excluding the additional languages from foreign learners it has become a daunting task to create tests that are fair across multilingual learners in one classroom. Items in tests that function differently from one group to another can provide biased marks. An investigation was done in order to detect any biased items present in a Picture Vocabulary Test. The study was lead by the main research question being: How do objects used in a Picture Vocabulary Test influence the level of validity? The first sub research question was: How do objects used in a Picture Vocabulary Test influence the level of validity? The next sub question was: To what extent is an undimensional trait measured by a Picture Vocabulary Test? The final subquestion was To what extent do the items in a Picture Vocabulary Test perform the same for the different language groups? This Picture Vocabulary Test was administered to Grade 1 learners in Afrikaans, English or Sepedi speaking schools within Pretoria, Gauteng. The sample totalling 1361 learners. The process involved a statistical procedure known as Rasch analyses. With the help of Rasch a Differential Item Functioning (DIF) analysis was done to investigate whether biased items were present in the test. The aim of this study it is to create greater awareness as to how biased items in tests can be detected and resolved. The results showed that the items in the Picture Vocabulary Test all tested vocabulary. Although items were detected that did indeed perform differently across the three language groups participating in the study. / Dissertation (MEd)--University of Pretoria, 2010. / Science, Mathematics and Technology Education / unrestricted

Multistage tests are a widely used and efficient type of test presentation that aims to provide accurate ability estimates while keeping the test relatively short. Multistage tests typically rely on the psychometric framework of item response theory. Violations of item response models and other assumptions underlying a multistage test, such as differential item functioning, can lead to inaccurate ability estimates and unfair measurements. There is a practical need for methods to detect problematic model violations to avoid these issues. This study compares and evaluates three methods for the detection of differential item functioning with regard to continuous person covariates in data from multistage tests: a linear logistic regression test and two adaptations of a recently proposed score-based DIF test. While all tests show a satisfactory Type I error rate, the score-based tests show greater power against three types of DIF effects.

No description available.

No description available.

No description available.