Back to search

University of South Florida

Effectiveness of Propensity Score Methods in a Multilevel Framework: A Monte Carlo Study

Abstract

dc:description

Propensity score analysis has been used to minimize the selection bias in observational studies to identify causal relationships. A propensity score is an estimate of an individual's probability of being placed in a treatment group given a set of covariates. Propensity score analysis aims to use the estimate to create balanced groups, akin to a randomized experiment. This study used Monte Carlo methods to examine the appropriateness of using propensity score methods to achieve balance between groups on observed covariates and reproduce treatment effect estimates in multilevel studies. Specifically, this study examined the extent to which four different propensity score estimation models and three different propensity score conditioning methods produced balanced samples and reproduced the treatment effects with clustered data. One single-level logistic model and three multilevel models were investigated. Conditioning methods included: (a) covariance adjustment, (b) matching, and (c) stratification. Design factors investigated included: (a) level-1sample size, (b) level-2 sample size, (c) level-1 covariate relationship to treatment, (d) level-2 covariate relationship to treatment, (e) level-1 covariate relationship to outcome, (f) level-2 covariate relationship to outcome, and (g) population effect size. The results of this study suggest the degree to which propensity score analyses are able to create balanced groups and reproduce treatment effect estimates with clustered data is largely dependent upon the propensity score estimation model and conditioning method selected. Overall, the single-level logistic and random intercepts models fared slightly better than the more complex multilevel models while covariance adjustment and matching methods tended to be more stable in terms of balancing groups than stratification. Additionally, the results indicate propensity score analysis should not be conducted with small samples. Finally, this study did not identify an estimation model or conditioning method that was consistently able to create adequately balanced groups and reproduce treatment effect estimates.

Degree

thesis:*
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Educational Measurement and Research
Grantor
University of South Florida
Year dc:date
2013

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Bellara, Aarti P.
Contributors dc:contributor
  • Jeffrey D. Kromrey

Subjects

dc:subject × 6

Rights

Language dc:language
EN-US

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/4635

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Bellara, Aarti P.. Effectiveness of Propensity Score Methods in a Multilevel Framework: A Monte Carlo Study. Dissertation thesis, University of South Florida, 2013. https://digitalcommons.usf.edu/etd/4635