No description available.

国立情報学研究所で電子化したコンテンツを使用している。

While assessing emotions, behaviors or performance of preschoolers and young children, scores from adults such as parent psychiatrist and teacher ratings are used rather scores from children themselves. Data from parent ratings or from parents and teachers are often nested such as students are within teachers and a child is within their parents. This popular nested feature of data in educational, social and behavioral sciences makes measurement invariance (MI) testing across informants of children methodologically challenging. There was lack of studies that take into account the nested structure of data in MI testing for multiple adult informants, especially no simulation study that examines the performance of different models used to test MI across different raters. This dissertation focused on two specific nesting data types in testing MI between adult raters of children: paired and partial nesting. For the paired data, the independence assumption of regular MI testing is often violated because the two informants (e.g., father and mother) rate the same child and their scores are anticipated to be related or dependent. The partial nesting data refers to the research situation where teacher and parent ratings are compared. In this scenario, it is common that each parent has only one child to rate while each teacher has multiple children in their classroom. Thus, in case of teacher and parent ratings of the same children, data are repeated measures and also partially nested. Because of these unique features of data, MI testing between adult informants of children requires statistical models that take into account different types of data dependency. I proposed and evaluated the performance of the two statistical models that can handle repeated measures and partial nesting with several simulated research scenarios in addition to one commonly used and one potentially appropriate statistical models across several research scenario. Results of the two simulation studies in this dissertation showed that for the paired data, both multiple-group confirmatory factor analysis (CFA) and repeated measure CFA models were able to detect scalar invariance most of the time using Δχ2 test and ΔCFI. Although the multiple-group CFA (Model 2) was able to detect scalar invariance better than the repeated measure CFA model (Model 1), the detection rates of Model 1 were still at the high level (88% - 91% using Δχ2 test and 84% - 100% using ΔCFI or ΔRMSEA). For configural invariance and metric invariance conditions for the paired data, Model 1 had higher detection rate than Model 2 in almost examined research scenario in this dissertation. Particularly while Model 1 could detect noninvariance (either in intercepts only or in both intercepts and factor loadings) than Model 2 for paired data most of the time, Model 2 could rarely catch it if using suggested cut-off of 0.01 for RMSEA differences. For the paired data, although both Models 1 and 2 could be a good choice to test measurement invariance, Model 1 might be favored if researchers are more interested in detecting noninvariance due to its overall high detection rates for all three levels (i.e. configural, metric, and scalar) of measurement invariance. For scalar invariance with partially nested data, both multilevel repeated measure CFA and design-based multilevel CFA could detect invariance most of the time (from 81% to 100% of examined cases) with slightly higher detection rate for the former model than the later. Multiple-group CFA model hardly detect scalar invariance except when ICC was small. The detection rates for configural invariance using Δχ2 test or Satorra-Bentler LRT were also highest for Model 3 (82% to 100% except only two conditions with detection rates of 61%), following by Model 5 and lowest Model 4. Models 4 and 5 could reach these rates only with the largest sample sizes (i.e., large number of cluster or large cluster size or large in both factors) when the magnitude of noninvariance was small. Unlike scalar and configural invariance, the ability to detect metric invariance was highest for Model 4, following by Model 5 and lowest for Model 3 across many conditions using all of the three performance criteria. As higher detection rates for all configural and scalar invariance, and moderate detection rates for many metric invariance conditions (except cases of small number of clusters combined with large ICC), Model 3 could be a good candidate to test measurement invariance with partially nested data when having sufficient number of clusters or if having small number of clusters with small ICC. Model 5 might be also a reasonable option for this type of data if both the number of clusters and cluster size were large (i.e., 80 and 20, respectively), or either one of these two factors was large coupled with small ICC. If ICC is not small, it is recommended to have a large number of clusters or combination of large number of clusters and large cluster size to ensure high detection rates of measurement invariance for partially nested data. As multiple group CFA had better and reasonable detection rates than the design-based and multilevel repeated measure CFA models cross configural, metric and scalar invariance with the conditions of small cluster size (10) and small ICC (0.13), researchers can consider using this model to test measurement invariance when they can only collect 10 participants within a cluster (e.g. students within a classroom) and there is small degree of data dependency (e.g. small variance between clusters) in the data.

This study involves investigating the impacts of ignoring nested data structure in Rasch/1PL item response theory (IRT) model via a two-level and three-level hierarchical generalized linear model (HGLM). Currently, Rasch/IRT models are frequently used in educational and psychometric researches for data obtained from multistage cluster samplings, which are more likely to violate the assumption of independent observations of examinees required by Rasch/IRT models. The violation of the assumption of independent observation, however, is ignored in the current standard practices which apply the standard Rasch/IRT for the large scale testing data. A simulation study (Study Two) was conducted to address this issue of the effects of ignoring nested data structure in Rasch/IRT models under various conditions, following a simulation study (Study One) to compare the performances of three methods, such as Penalized Quasi-Likelihood (PQL), Laplace approximation, and Adaptive Gaussian Quadrature (AGQ), commonly used in HGLM in terms of accuracy and efficiency in estimating parameters. As expected, PQL tended to produce seriously biased item difficulty estimates and ability variance estimates whereas almost unbiased for Laplace or AGQ for both 2-level and 3-level analysis. As for the root mean squared errors (RMSE), three methods performed without substantive differences for item difficulty estimates and ability variance estimates in both 2-level and 3-level analysis, except for level-2 ability variance estimates in 3-level analysis. Generally, Laplace and AGQ performed similarly well in terms of bias and RMSE of parameter estimates; however, Laplace exhibited a much lower convergence rate than that of AGQ in 3-level analyses. The results from AGQ, which produced the most accurate and stable results among three computational methods, demonstrated that the theoretical standard errors (SE), i.e., asymptotic information-based SEs, were underestimated by at most 34% when 2-level analyses were used for the data generated from 3-level model, implying that the Type I error rate would be inflated when the nested data structures are ignored in Rasch/IRT models. The underestimated theoretical standard errors were substantively more severe as the true ability variance increased or the number of students within schools increased regardless of test length or the number of schools. / Ph. D.

The literature is clear that intentional or unintentional clustering of data elements typically results in the inflation of the estimated standard error of fixed parameter estimates. This study is unique in that it examines the impact of multilevel data structures on subject ability which are random effect predictions known as empirical Bayes estimates in the one-parameter IRT / Rasch model. The literature on the impact of complex survey design on latent trait models is mixed and there is no "best practice" established regarding how to handle this situation. A simulation study was conducted to address two questions related to ability estimation. First, what impacts does design based clustering have with respect to desirable statistical properties when estimating subject ability with the one-parameter IRT / Rasch model? Second, since empirical Bayes estimators have shrinkage properties, what impacts does clustering of first-stage sampling units have on measurement validity-does the first-stage sampling unit impact the ability estimate, and if so, is this desirable and equitable? Two models were fit to a factorial experimental design where the data were simulated over various conditions. The first model Rasch model formulated as a HGLM ignores the sample design (incorrect model) while the second incorporates a first-stage sampling unit (correct model). Study findings generally showed that the two models were comparable with respect to desirable statistical properties under a majority of the replicated conditions-more measurement error in ability estimation is found when the intra-class correlation is high and the item pool is small. In practice this is the exception rather than the norm. However, it was found that the empirical Bayes estimates were dependent upon the first-stage sampling unit raising the issue of equity and fairness in educational decision making. A real-world complex survey design with binary outcome data was also fit with both models. Analysis of the data supported the simulation design results which lead to the conclusion that modeling binary Rasch data may resort to a policy tradeoff between desirable statistical properties and measurement validity. / Ph. D.

This study compared 3 approaches for handling a fourth level of nesting structure when analyzing data from a cluster-randomized trial (CRT). CRTs can include 3 levels of nesting: repeated measures, individual, and cluster levels. However, above the cluster level, there may sometimes be an additional potentially important fourth level of nesting (e.g., schools, districts, etc., depending on the design) that is typically ignored in CRT data analysis. The current study examined the impact of ignoring this fourth level, accounting for it using a model-based approach, and accounting it using a design-based approach on parameter and standard error (SE) estimates. Several fixed effect and random effect variance parameters and SEs were biased across all 3 models. In the 4-level model, most SE biases decreased as the number of level 3 clusters increased and as the number of level 4 clusters decreased. Also, random effect variance biases decreased as the number of level 3 clusters increased. In the 3-level and complex models, SEs became more biased as the weight level 4 carried increased (i.e., larger intraclass correlation, more clusters at that level). The current results suggest that if a meaningful fourth level of nesting exists, future researchers should account for it using design-based approach; the model-based approach is not recommended. If the fourth level is not practically important, researchers may ignore it altogether.

This study aims to identify and explain relationships between some major factors associated with successful reading at Grade 5 level in South African primary schools. In South Africa, grave concerns with regards to low levels of student achievement pervade research initiatives and educational debates. Despite considerable investments in educational inputs (such as policy and resources) and processes (such as curriculum provision and teacher support), outcomes (such as student achievement) remain disappointingly low. The South African population is characterized by great diversity and variation. With 11 official languages, current educational policy in South Africa advocates an additive bilingualism model and students in Grade 1 to 3 are taught in their mother tongue. Thereafter, when these students progress to Grade 4, the language of learning and teaching changes to a second language, which in most cases is English. At this key developmental stage students are also expected to advance from learning to read to a stage where they can use reading in order to learn. With this complexity of issues in mind, Hierarchical Linear Modeling (HLM) was used to determine the effect of a number of explanatory variables at learner and school level on reading achievement as outcome variable, while controlling for language using the South African Progress in International Reading Literacy Study (PIRLS) 2006 data. As an international comparative evaluation of reading literacy involving more than 40 countries, PIRLS 2006 was the second, after PIRLS 2001, in a series of planned five-year cycles of assessment to measure trends in children’s reading literacy achievement, policy and practices related to literacy. Grade 5 learners in South African primary schools who participated in PIRLS 2006 were not able to achieve satisfactory levels of reading competence. The gravity of this finding is exacerbated by the fact that these learners were tested in the language in which they had been receiving instruction during the Foundation Phase of schooling. This study found most significant factors associated with reading literacy at learner-level, but this does not mean that the existence of teacher- and school-level factors is not of importance. While some explanatory factors at learner-level can more easily become the target of reading interventions, the higher level effect of the classroom and school are not diminished by this study. Creemers’ Comprehensive Model of Educational Effectiveness was utilized as theoretical point of departure. Creemers’ model was adapted for the purposes of this study to reflect a South African model of reading effectiveness in contrast with Creemers’ original use of it as a model of school effectiveness. Evidence was provided that the conceptual framework was inadequate in identifying factors affecting reading achievement for all South African language groupings. More specifically, the adapted South African reading effectiveness model was only appropriate in explaining reading achievement scores for the Afrikaans and English language groupings than for those from African language groupings. / Thesis (PhD)--University of Pretoria, 2010. / Science, Mathematics and Technology Education / unrestricted

No description available.