Back to search

Virginia Tech

Examining Lord's Paradox from Causal Inference Perspective: A Simulation Study

Abstract

dc:description.abstract

In two-wave nonequivalent control group designs, Change Score Analysis (CSA) and Analysis of Covariance (ANCOVA) are commonly used to estimate treatment effects. However, these analytical models can yield conflicting results in both direction and magnitude – a phenomenon known as Lord's Paradox – raising concerns regarding analytical method selection in non-experimental research. This dissertation examines Lord's Paradox from a causal inference perspective to clarify the conditions under which these discrepancies arise. Specifically, the study aims to identify the conditions under which CSA and ANCOVA yield unbiased estimates, to determine when their results converge or diverge, and to assess how measurement error in the pretest affects their performance. To address these objectives, a general data-generating model is employed that incorporates unobserved confounding variable. Analytical derivations are conducted to express the CSA and ANCOVA estimators as functions of structural parameters, enabling a precise characterization of bias. These results are complemented by simulation studies that evaluate estimator performance in several metrics. The findings indicate that CSA yields unbiased estimates only under restrictive conditions, particularly in the absence of dynamic selection or symmetric confounding (i.e., when the effects of confounding variable on the pretest and the posttest are equal). In contrast, ANCOVA achieves unbiasedness under broader conditions, although it may also produce biased estimates when its assumptions are violated. Divergence between CSA and ANCOVA results is shown to be common in the presence of dynamic selection and asymmetric confounding, and may involve not only differences in magnitude but also reversals in the direction of the estimated effect. The analysis further demonstrates that measurement error in the pretest substantially affects estimator performance, particularly for ANCOVA, as declining reliability leads to increased bias and reduced inferential accuracy.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Educational Research and Evaluation
Department dc:contributor.department
Educational Research and Evaluation
Grantor dc:publisher
Virginia Tech
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Zhu, Xiao
Chair dc:contributor.committeechair
  • Miyazaki, Yasuo
Committee members dc:contributor.committeemember
  • Kniola, David John
  • Jones, Brett D.
  • Johnson, Marcus L.

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • In Copyright
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Other
vt_gsexam:46414

Chain of custody

source
Harvested from
Virginia Tech
Base URL
vtechworks.lib.vt.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Zhu, Xiao. Examining Lord's Paradox from Causal Inference Perspective: A Simulation Study. doctoral thesis, Virginia Tech, 2026. https://hdl.handle.net/10919/143433