Back to search

University of Central Florida

A Priori Analysis of Error and Bias in Value-Added Models

Abstract

dc:description.abstract

Over the past 20 years, value-added models (VAMs) have become increasingly popular in educational assessment and accountability policies because of the sophisticated statistical controls these models use to purportedly isolate the effect of a single teacher on the learning gains of his or her students. The present research uses a Monte Carlo simulation study design in order to investigate whether VAMs are able to provide accurate estimates of teacher effectiveness when all assumptions are met and to determine how robust the models are to endogenous peer effects and nonrandom assignment of students to classroom. The researcher generates three years of simulated achievement data for 18,750 students taught by 125 teachers, and analyzes this data with a linear mixed model similar to the SAS EVAAS Multivariate Response Model (MRM; M1), a basic covariate adjustment model (M2), and variations on these models designed to estimate random classroom effects. Findings indicate that the modified EVAAS may be too computationally onerous to be of practical use, and that modified covariate adjustment models do not perform significantly differently than the basic covariate adjustment model. When all assumptions are met, M1 is more accurate than M2, but both models perform reasonably well, misclassifying fewer than 5% of teachers on average. M1 is more robust to endogenous peer effects than M2, however both models misclassified more teachers than when all assumptions are met. M2 is more robust to nonrandom assignment of students than M1. Assigning teachers a balanced schedule of nonrandom classes with low, medium, and high prior achievement seemed to mitigate the problems that nonrandom assignment caused for M1, but made M2 less accurate. Implications for practice and future research are discussed.

Degree

thesis:*
Grantor dc:publisher
University of Central Florida

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lavery, Matthew
Contributors dc:contributor
  • Hahs-Vaughn, Debbie

Subjects

dc:subject × 2

Rights

Language dc:language
English

Identifiers

dc:identifier.*
Identifier
CFE0006344

Chain of custody

source
Harvested from
Central Florida
Base URL
stars.library.ucf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Lavery, Matthew. A Priori Analysis of Error and Bias in Value-Added Models. University of Central Florida, https://stars.library.ucf.edu/etd/5213