West Virginia University
Influences of Response Rate and Distribution on Accuracy Estimates: The Value of Replications Based on Analytic Approach
Abstract
dc:description.abstractAccurate observational data are essential for drawing correct conclusions about behavioral phenomena and ensuring the replicability of research findings. Accuracy is the extent to which an observer’s recorded responses correspond with true values (i.e., the actual occurrence of behavioral events). Correspondence is quantified using one of several percentage-based algorithms. However, these accuracy estimates may be influenced by characteristics of the data, such as response rate and response distribution, as well as the algorithm used to calculate them. The purpose of this dissertation was to evaluate how response rate, response distribution, and the algorithm used to evaluate accuracy influences estimates. Across both experiments, participants collected continuous data from videos of mock teacher-student interactions using computerized data-collection software, and their records were compared with true values. Experiment 1 assessed the effects of response rate on accuracy estimates across a continuous spectrum of rates, four algorithms, and a single response topography. A mixed effects regression with a polynomial term found that response rate differentially influenced each algorithm. Experiment 2 evaluated the effects of response distribution on accuracy estimates across eight algorithms, including occurrence and nonoccurrence variants, and identified the types of data-collection errors detected by each algorithm. An ANOVA identified a significant interaction between distribution and algorithm and significant main effects for distribution and algorithm. Collectively, the findings demonstrate that accuracy estimates are not solely a reflection of an observer’s skills, but are influenced by response rate, response distribution, and the algorithm used to calculate them. Researchers and practitioners should interpret accuracy estimates in the context of both the characteristics of the data and the algorithms used to calculate them to meaningfully evaluate the believability of observational data.
Degree
thesis:*- Name thesis:degree_name
- PhD
- Level thesis:degree_level
- Dissertation
- Discipline thesis:degree_discipline
- Psychology
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Harvey, Olivia Brianne
- Contributors dc:contributor
-
- Claire St. Peter
- Kathryn Kestner
- Kathleen Morrison
- Ryan Best
- Stephanie Jones
Subjects
dc:subject × 6Identifiers
dc:identifier.*- Identifier
-
<p><a href="https://orcid.org/0000-0001-7323-7250">https://orcid.org/0000-0001-7323-7250</a></p>
https://researchrepository.wvu.edu/etd/13463