Back to search

University of South Florida

Statistical Challenges and Simulation Results for Pilot Clinical Trials

Abstract

dc:description

Background: The effect size estimated from a pilot trial is often an inaccurate reflection of the true effect size observed in a large trial, leading to either underestimation or overestimation. Published data suggest that effect sizes from large trials are typically smaller than those reported in their corresponding pilot trials. To address this discrepancy, conservative or discount adjustment methods are widely recommended to modify pilot trial effect sizes when calculating sample sizes, thereby maintaining adequate statistical power. This study aims to assess effect sizes from both pilot and large trials and to evaluate the performance of existing adjustment methods. Methods: Published effect sizes from both pilot trials and large trials were systematically reviewed. Existing adjustment methods for pilot trial effect sizes, including multiplicative, additive, lower confidence limit (LCL), and the Dual-Bound Weighted method, were evaluated through simulated clinical trials across a range of true effect sizes and sample sizes. Both underestimation and overestimation were examined and compared. Results: When pilot estimates showed upward bias, conservative methods reduced overestimation and improved accuracy. However, when pilot trials underestimated the true effect, these methods exaggerated underestimation, leading to excessively large sample size recommendations. Performance varied by context: multiplicative and 60% LCL adjustments worked well for small and moderate effects under random or upward bias, but none were optimal across all scenarios. The proposed DB-weighted method achieved greater balance and accuracy for larger sample sizes and effect sizes, regardless of bias direction. Conclusion: Findings challenge the universal use of conservative adjustments of pilot ES estimates. Researchers should tailor adjustment strategies based on bias direction, sample size, and effect size magnitude. The DB-weighted method provides a more balanced alternative when bias direction is uncertain. This study offers practical guidance for selecting adjustment methods based on suspected bias direction, sample size, and effect size magnitude.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (Ph.D.)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Public Health
Grantor
University of South Florida
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Cen, Weiliang
Contributors dc:contributor
  • Henian Chen, Ph.D.
  • Stephanie Marhefka-Day, Ph.D.
  • Wei Wang, Ph.D.
  • Yangxin Huang, Ph.D.
  • Matthew Valente, Ph.D.

Subjects

dc:subject × 7

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/11077

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Cen, Weiliang. Statistical Challenges and Simulation Results for Pilot Clinical Trials. Dissertation thesis, University of South Florida, 2025. https://digitalcommons.usf.edu/etd/11077