This study applied Monte Carlo simulations to investigate the robustness of parameter estimates for a two-level Hierarchical Linear Model (HLM) to the violations of the second-level residual homoskedasticity and independence assumptions. It considered these violations in the context of a typical HLM model with random intercepts as outcomes, a model that has been commonly used in value-added analysis for school settings. The study had a factorial design with unbalanced data, where five factors were considered in the simulations. The first two represented variance and independence assumption violations, while the other three were conditional. The baseline values for the model specifications and sample size reflected results obtained from HLM analysis conducted on an actual school data system. Variations of these values were considered in representing the five study factors. The first factor represented three homoskedasticity levels of the residuals at the second level of the model: no assumption violation, moderate violation, and an extreme violation. The second factor represented three independence levels for the residuals at the second level of the model: no assumption violation, a violation by misspecifing a second-level predictor having a medium-positive effect size, and a violation by misspecifing a second-level predictor having a small-negative effect size. The other three factors in the study included three levels of collinearity among second-level predictors; two sample sizes at the second level; and two average sample sizes at the first level. The five factors manipulated in this study produced 108 conditions, and 100 replications were used for each condition. The robustness of the HLM parameter estimates was evaluated through the absolute and relative bias, the mean square error, and the absolute and relative inaccuracy of their apparent standard errors. In addition, a five-way factorial ANOVA was used to identify significant and relatively important main and interaction effects of the study factors on the bias of the estimates for each parameter. In addition, graphs were utilized mainly to show variations in the accuracy of the apparent precision of the estimates for each parameter under the study conditions. The study results showed that the HLM parameter estimates and their apparent precision were robust to moderate and severe differences in second-level residual variances. However, they relied heavily on having independent second-level residuals, resulting from a correctly specified model. When the independence assumption of the second-level residuals was violated, the HLM parameter estimates of the second-level predictors and variance were biased and the accuracy of the apparent precision of all HLM second-level parameter estimates decreased. The effect of this violation became more evident as sample size was decreased, particularly at the second level. This effect depended also on the parameter of interest, the correlation of the misspecified predictor with other predictors included in the model, in addition to the effect size and sign of this misspecified predictor. When the assumption violation factors interacted with the conditional factors, collinearity level and sample sizes, the pattern of their effects on bias and accuracy of apparent precision differed from their pattern when they were considered separately. More specifically, the pattern of interaction was considerable for the independence assumption violations. The results also showed that the apparent standard errors produced by HLM had a tendency to be larger than the actual standard errors for all parameters, particularly with small sample sizes. This could be partially attributed to the correlation among the predictors at the different levels of the model and centering-related issues. / A Dissertation submitted to the Department of Educational Psychology and Learning
Systems in partial fulfillment of the requirements for the degree of Doctor of
Philosophy. / Degree Awarded: Spring Semester, 2004. / Date of Defense: December 12, 2003. / Multilevel Models, Level-2 Asssumptions, School Effectiveness, Value-added Model Full Maximum Likelihood Error Variance / Includes bibliographical references. / Richard L. Tate, Professor Directing Dissertation; Colleen Kelley, Outside Committee Member; Albert Oosterhof, Committee Member; Akihito Kamata, Committee Member.
| Identifer | oai:union.ndltd.org:fsu.edu/oai:fsu.digital.flvc.org:fsu_169031 |
| Contributors | Darandari, Eqbal Z. (authoraut), Tate, Richard L. (professor directing dissertation), Kelley, Colleen (outside committee member), Oosterhof, Albert (committee member), Kamata, Akihito (committee member), Department of Educational Psychology and Learning Systems (degree granting department), Florida State University (degree granting institution) |
| Publisher | Florida State University |
| Source Sets | Florida State University |
| Language | English, English |
| Detected Language | English |
| Type | Text, text |
| Format | 1 online resource, computer, application/pdf |
Page generated in 0.002 seconds