While use of hierarchical linear modeling (HLM) to predict an outcome is reasonable and desirable, employing the model for prediction without first establishing the model’s predictive validity is ill-advised. Estimating the predictive validity of a regression model by cross-validation has been thoroughly researched, but there is a dearth of research investigating the cross-validation of hierarchical linear models. One of the major obstacles in cross-validating HLM is the lack of a measure of explained variance similar to the squared multiple correlation coefficient in regression analysis. The purpose of this Monte Carlo simulation study is to explore the impact of sample size, centering, and predictor-criterion correlation magnitudes on potential cross-validation measurements for hierarchical linear modeling. This study considered the impact of 64 simulated conditions across three explained variance approaches: Raudenbush and Bryk’s (2002) proportional reduction in error variance, Snijders and Bosker’s (1994) modeled variance, and a measure of explained variance proposed by Gagné and Furlow (2009). For each of the explained variance approaches, a cross-validation measurement, shrinkage, was obtained. The results indicate that sample size, predictor-criterion correlations, and centering impact the cross-validation measurement. The degree and direction of the impact differs with the explained variance approach employed. Under some explained variance approaches, shrinkage decreased with larger level-2 sample sizes and increased in others. Likewise, in comparing group- and grand-mean centering, with some approaches grand-mean centering resulted in higher shrinkage estimates but smaller estimates in others. Larger total sample sizes yielded smaller shrinkage estimates, as did the predictor-criterion correlation combination in which the group-level predictor had a stronger correlation. The approaches to explained variance differed substantially in their usability for cross-validation. The Snijders and Bosker approach provided relatively large shrinkage estimates, and, depending on the predictor-criterion correlation, shrinkage under both Raudenbush and Bryk approaches could be sizable to the degree that the estimate begins to lack meaning. Researchers seeking to cross-validate HLM need to be mindful of the interplay between the explained variance approach employed and the impact of sample size, centering, and predictor-criterion correlations on shrinkage estimates when making research design decisions.

This study aims to identify and explain relationships between some major factors associated with successful reading at Grade 5 level in South African primary schools. In South Africa, grave concerns with regards to low levels of student achievement pervade research initiatives and educational debates. Despite considerable investments in educational inputs (such as policy and resources) and processes (such as curriculum provision and teacher support), outcomes (such as student achievement) remain disappointingly low. The South African population is characterized by great diversity and variation. With 11 official languages, current educational policy in South Africa advocates an additive bilingualism model and students in Grade 1 to 3 are taught in their mother tongue. Thereafter, when these students progress to Grade 4, the language of learning and teaching changes to a second language, which in most cases is English. At this key developmental stage students are also expected to advance from learning to read to a stage where they can use reading in order to learn. With this complexity of issues in mind, Hierarchical Linear Modeling (HLM) was used to determine the effect of a number of explanatory variables at learner and school level on reading achievement as outcome variable, while controlling for language using the South African Progress in International Reading Literacy Study (PIRLS) 2006 data. As an international comparative evaluation of reading literacy involving more than 40 countries, PIRLS 2006 was the second, after PIRLS 2001, in a series of planned five-year cycles of assessment to measure trends in children’s reading literacy achievement, policy and practices related to literacy. Grade 5 learners in South African primary schools who participated in PIRLS 2006 were not able to achieve satisfactory levels of reading competence. The gravity of this finding is exacerbated by the fact that these learners were tested in the language in which they had been receiving instruction during the Foundation Phase of schooling. This study found most significant factors associated with reading literacy at learner-level, but this does not mean that the existence of teacher- and school-level factors is not of importance. While some explanatory factors at learner-level can more easily become the target of reading interventions, the higher level effect of the classroom and school are not diminished by this study. Creemers’ Comprehensive Model of Educational Effectiveness was utilized as theoretical point of departure. Creemers’ model was adapted for the purposes of this study to reflect a South African model of reading effectiveness in contrast with Creemers’ original use of it as a model of school effectiveness. Evidence was provided that the conceptual framework was inadequate in identifying factors affecting reading achievement for all South African language groupings. More specifically, the adapted South African reading effectiveness model was only appropriate in explaining reading achievement scores for the Afrikaans and English language groupings than for those from African language groupings. / Thesis (PhD)--University of Pretoria, 2010. / Science, Mathematics and Technology Education / unrestricted