Return to search

Beyond the Turk: Alternative platforms for crowdsourcing behavioral research

The success of Amazon Mechanical Turk (MTurk) as an online research platform has come at a price: MTurk has suffered from slowing rates of population replenishment, and growing participant non-naivety. Recently, a number of alternative platforms have emerged, offering capabilities similar to MTurk but providing access to new and more naïve populations. After surveying several options, we empirically examined two such platforms, CrowdFlower (CF) and Prolific Academic (ProA). In two studies, we found that participants on both platforms were more naïve and less dishonest compared to MTurk participants. Across the three platforms, CF provided the best response rate, but CF participants failed more attention-check questions and did not reproduce known effects replicated on ProA and MTurk. Moreover, ProA participants produced data quality that was higher than CF's and comparable to MTurk's. ProA and CF participants were also much more diverse than participants from MTurk.

Identiferoai:union.ndltd.org:arizona.edu/oai:arizona.openrepository.com:10150/623545
Date05 1900
CreatorsPeer, Eyal, Brandimarte, Laura, Samat, Sonam, Acquisti, Alessandro
ContributorsEller College of Management, University of Arizona
PublisherACADEMIC PRESS INC ELSEVIER SCIENCE
Source SetsUniversity of Arizona
LanguageEnglish
Detected LanguageEnglish
TypeArticle
Rights© 2017 Elsevier Inc. All rights reserved.
Relationhttp://linkinghub.elsevier.com/retrieve/pii/S0022103116303201, https://osf.io/g3c56/

Page generated in 0.0022 seconds