Back to search

University of British Columbia

Understanding, evaluating, and justifying sample size : a systematic review and meta-meta-analysis

Abstract

dc:description

The pursuit of explicitly justifying a study’s sample size began in hopes of mitigating p-hacking and false positive rates across psychology through having researchers be more thoughtful about the research process. However, there is still the question about how to properly justify a sample size for a study. A review of five years of sample size justifications in Psychological Science demonstrates that researchers use many different methods to justify a sample’s size. For example, some common justifications are heuristics, resource constraints, and priori power analysis. While there are many ways to justify a study’s sample size, much uncertainty still surrounds the merit and validity of the different types of sample size justifications. To give researchers and readers a better benchmark to justify and evaluate sample size, I propose using the distribution of effect sizes in a specific discipline to find the average statistical power, known as generalized average power. Starting with the distribution of effect sizes in social psychology, I use robust Bayesian model averaging across 71 meta-analyses to simulate the distribution of population effect sizes in social psychology. I then use the distribution to determine the sample size needed to maintain an average power of 80% across the field. This provides an evidence-based heuristic for researchers, readers, reviewers, and editors to effectively evaluate — and justify — sample sizes.

Degree

thesis:*
Name thesis:degree_name
Master of Arts - MA
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Psychology
Grantor dc:publisher
University of British Columbia
Year dc:date
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lemmons, Malina C.
Contributors dc:contributor
  • Biesanz, Jeremy C.

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/88880

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Lemmons, Malina C.. Understanding, evaluating, and justifying sample size : a systematic review and meta-meta-analysis. master's thesis, University of British Columbia, 2024. http://hdl.handle.net/2429/88880