University of South Florida
Statistical Challenges and Simulation Results for Pilot Clinical Trials
Abstract
dc:descriptionBackground: The effect size estimated from a pilot trial is often an inaccurate reflection of the true effect size observed in a large trial, leading to either underestimation or overestimation. Published data suggest that effect sizes from large trials are typically smaller than those reported in their corresponding pilot trials. To address this discrepancy, conservative or discount adjustment methods are widely recommended to modify pilot trial effect sizes when calculating sample sizes, thereby maintaining adequate statistical power. This study aims to assess effect sizes from both pilot and large trials and to evaluate the performance of existing adjustment methods. Methods: Published effect sizes from both pilot trials and large trials were systematically reviewed. Existing adjustment methods for pilot trial effect sizes, including multiplicative, additive, lower confidence limit (LCL), and the Dual-Bound Weighted method, were evaluated through simulated clinical trials across a range of true effect sizes and sample sizes. Both underestimation and overestimation were examined and compared. Results: When pilot estimates showed upward bias, conservative methods reduced overestimation and improved accuracy. However, when pilot trials underestimated the true effect, these methods exaggerated underestimation, leading to excessively large sample size recommendations. Performance varied by context: multiplicative and 60% LCL adjustments worked well for small and moderate effects under random or upward bias, but none were optimal across all scenarios. The proposed DB-weighted method achieved greater balance and accuracy for larger sample sizes and effect sizes, regardless of bias direction. Conclusion: Findings challenge the universal use of conservative adjustments of pilot ES estimates. Researchers should tailor adjustment strategies based on bias direction, sample size, and effect size magnitude. The DB-weighted method provides a more balanced alternative when bias direction is uncertain. This study offers practical guidance for selecting adjustment methods based on suspected bias direction, sample size, and effect size magnitude.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy (Ph.D.)
- Level thesis:degree_level
- Dissertation
- Discipline thesis:degree_discipline
- Public Health
- Grantor
- University of South Florida
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Cen, Weiliang
- Contributors dc:contributor
-
- Henian Chen, Ph.D.
- Stephanie Marhefka-Day, Ph.D.
- Wei Wang, Ph.D.
- Yangxin Huang, Ph.D.
- Matthew Valente, Ph.D.
Subjects
dc:subject × 7Identifiers
dc:identifier.*- Repository record dc:identifier
- https://digitalcommons.usf.edu/etd/11077