University of British Columbia
Understanding, evaluating, and justifying sample size : a systematic review and meta-meta-analysis
Abstract
dc:descriptionThe pursuit of explicitly justifying a study’s sample size began in hopes of mitigating p-hacking and false positive rates across psychology through having researchers be more thoughtful about the research process. However, there is still the question about how to properly justify a sample size for a study. A review of five years of sample size justifications in Psychological Science demonstrates that researchers use many different methods to justify a sample’s size. For example, some common justifications are heuristics, resource constraints, and priori power analysis. While there are many ways to justify a study’s sample size, much uncertainty still surrounds the merit and validity of the different types of sample size justifications. To give researchers and readers a better benchmark to justify and evaluate sample size, I propose using the distribution of effect sizes in a specific discipline to find the average statistical power, known as generalized average power. Starting with the distribution of effect sizes in social psychology, I use robust Bayesian model averaging across 71 meta-analyses to simulate the distribution of population effect sizes in social psychology. I then use the distribution to determine the sample size needed to maintain an average power of 80% across the field. This provides an evidence-based heuristic for researchers, readers, reviewers, and editors to effectively evaluate — and justify — sample sizes.
Degree
thesis:*- Name thesis:degree_name
- Master of Arts - MA
- Level thesis:degree_level
- master's
- Discipline thesis:degree_discipline
- Psychology
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Lemmons, Malina C.
- Contributors dc:contributor
-
- Biesanz, Jeremy C.
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/88880