Back to search

Virginia Tech

A Graphical Approach to Identifying Structural Bias Using Directed Acyclic Graphs: Its Application to Two-Wave Nonequivalent Control Group Designs

Abstract

dc:description.abstract

It is well known that the analysis of covariance (ANCOVA) and the change-score analysis (CSA) can produce quite different treatment-effect estimates when applied to data from two-wave nonequivalent control-group designs, a phenomenon known as the Lord's paradox. Pearl's (2009) structural causal model (SCM) provides a useful and intuitively appealing tool to address the Lord's paradox. Using the SCM, Kim and Steiner (2021) combined the backdoor criterion with the path-tracing rules and showed that it identified the exact bias for the CSA. Though they implied that this graphical causal model approach could be applied to the ANCOVA case in a similar way, they did not explicitly show the details. Therefore, in the present study, to examine their implication, I applied the graphical approach to the ANCOVA and compared the results with the bias derived by the population ordinary least squares (OLS) method (Lüdtke and Robitzsch, 2025). The comparison exhibited a discrepancy, though the core part of the bias obtained by the graphical approach was correct. Specifically, the discrepancy occurred in the terms that were proportional to the core part of the bias implied by each backdoor path. This means that, though the detection of the sources of bias and the identification of the conditions to eliminate the bias could be completed by the graphical approach, the exact quantification of the bias was not possible. To resolve this shortcoming, I applied the so-called regression anatomy formula, also known as the Frisch–Waugh–Lovell (FWL) theorem in econometrics, and found that the proportional term could be expressed as the residualization-induced scaling factor. I then extended this graphical approach to different data-generating scenarios within two-wave nonequivalent control-group designs and confirmed that it worked well in all cases. The residualization procedure makes a graphical approach self-contained to identify the exact structural bias.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Educational Research and Evaluation
Department dc:contributor.department
Educational Research and Evaluation
Grantor dc:publisher
Virginia Tech
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Shin, Jaehyun
Chair dc:contributor.committeechair
  • Miyazaki, Yasuo
Committee members dc:contributor.committeemember
  • Jones, Brett D.
  • Kniola, David John
  • Hori, Kazuki

Subjects

dc:subject × 5

Rights

dc:rights
Statement dc:rights
  • In Copyright
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Other
vt_gsexam:45698

Chain of custody

source
Harvested from
Virginia Tech
Base URL
vtechworks.lib.vt.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Shin, Jaehyun. A Graphical Approach to Identifying Structural Bias Using Directed Acyclic Graphs: Its Application to Two-Wave Nonequivalent Control Group Designs. doctoral thesis, Virginia Tech, 2026. https://hdl.handle.net/10919/142387