No description available.

Crowdsourcing and microtasks are a relatively new way to issue units of work to a large group of potential workers. This form of outsourcing to a vast on-demand workforce offers the potential to significantly change the way we work. But how can design impact how both the requester and the workforce interact and benefit from these tasks? This dissertation considers four aspects of microtask design: value, engagement, context, and complexity. Through four distinct, but highly related, investigations these four facets are ex- plored, analysed and synthesised into a considered review of microtask design. First we build a picture of the demographic and financial status of these crowdworkers by surveying the US-based crowdworker labour-force on the Amazon Mechanical Turk platform. This improved understanding of the value of crowd work, not just to requesters but to workers as well, is crucial to appropriately listing tasks in a commoditised labour market. Second, worker engagement is also a significant factor, not just in quality and cost, but also in uptake and effective completion speed. By introducing a new metric, conversion rate, and contrasting a variety of differing presentational and conceptual features across two demographics, we demonstrate an improved understanding of how tasks engage workers. The increasing use of mobile devices, including among crowdworkers, offers new opportunities to collect additional context about worker behaviour. Enhancing the data gathered by requesters can be used, not only to improve quality, but also to expand the types of tasks which can be effectively crowdsourced. This third contribution highlights enthusiasm by some workers for mobile tasks, and demon- strates how previously small-scale sensor-based data collection can increasingly be carried out by the crowd. Finally, the boundary between microtasks and macrotasks is investigated. Exploring how complex tasks, such as software development, can be successfully crowdsourced offers insight into how task design can influence suitability of these larger tasks on microtask markets.

The success of Amazon Mechanical Turk (MTurk) as an online research platform has come at a price: MTurk has suffered from slowing rates of population replenishment, and growing participant non-naivety. Recently, a number of alternative platforms have emerged, offering capabilities similar to MTurk but providing access to new and more naïve populations. After surveying several options, we empirically examined two such platforms, CrowdFlower (CF) and Prolific Academic (ProA). In two studies, we found that participants on both platforms were more naïve and less dishonest compared to MTurk participants. Across the three platforms, CF provided the best response rate, but CF participants failed more attention-check questions and did not reproduce known effects replicated on ProA and MTurk. Moreover, ProA participants produced data quality that was higher than CF's and comparable to MTurk's. ProA and CF participants were also much more diverse than participants from MTurk.

No description available.

This study sought to determine if time spent engaging in online gaming, interpersonal communication skills, and introvert personality traits are predictors of an individual’s likelihood of experiencing symptoms of social anxiety. A sample of 128 participants (82 males and 46 females) completed measures of demographics, interpersonal communication skills, problematic online gaming, social anxiety, and introversion. Participants were recruited through Amazon Mechanical Turk. There were significant correlations among social anxiety and interpersonal communication skills, problematic online gaming, and introversion. There was no significant correlation among social anxiety and time spent playing Massively Multiplayer Online Role-Playing Games (MMORPG), a specific form of online video game. It was also found that interpersonal communication skills, problematic online gaming, and introversion were all significant predictors of social anxiety. However, time spent playing MMORPGs was not a significant predictor of social anxiety.

Excerpt: Time and expense are perhaps the two biggest challenges in evaluating existing measures and devoloping new metrics. Measuring social characterists of a population such as religion typically involves expensive surveys undertaken by professional survey firms or academic centers.

This dissertation investigates whether transitivity distinctions are manifest in the phonetics of linguistic and paralinguistic manual actions, namely lexical verbs and classifier constructions in American Sign Language (ASL) and gestures produced by hearing non-signers without speech (i.e., pantomime). A positive result would indicate that grammatical features are (a) transparent and (b) may thus arise from non-linguistic sources, here the visual-praxic domain. Given previous literature, we predict that transitivity is transparent in pantomime and classifier constructions, but opaque in lexical verbs. <div><br></div><div>We first collected judgments from hearing non-signers who classed pantomimes, classifier constructions, and ASL lexical verbs as unergative, unaccusative, transitive, or ditransitive. We found that non-signers consistently judged items across all three stimulus types, suggesting that there is transitivity-related information in the signed signal. </div><div><br></div><div>We then asked whether non-signers’ judging ability has its roots in a top-down or bottom-up strategy. A top-down strategy might entail guessing the meaning of the sign or pantomime and then using the guessed meaning to assess/guess its transitivity. A bottom-up strategy entails using one or more meaningful phonetic features available in the formation of the signal to judge an item. We predicted that both strategies would be available in classing pantomimes and classifier constructions, but that transitivity information would only be available top-down in lexical verbs, given that the former are argued to be more imagistic generally than lexical verbs. Further, each strategy makes a different prediction with respect to the internal representation xv of signs and pantomimes. The top-down strategy would suggest signs and pantomimes are unanalyzable wholes, whereas the bottom-up strategy would suggest the same are compositional. </div><div><br></div><div>For the top-down analysis, we correlated lexical iconicity score and a measure of the degree to which non-signers ‘agreed’ on the transitivity of an item. We found that lexical iconicity only weakly predicts non-signer judgments of transitivity, on average explaining 10-20% of the variance for each stimulus class. However, we note that this is the only strategy available for lexical verbs. </div><div><br></div><div>For the bottom-up analysis, we annotate our stimuli for phonetic and phonological features known to be relevant to transitivity and/ or event semantics in sign languages. We then apply a text classification model to try to predict transitivity from these features. As expected, our classifiers achieved stably high accuracy for pantomimes and classifier constructions, but only chance accuracy for lexical verbs. </div><div><br></div><div>Taken together, the top-down and bottom-up analyses were able to predict nonsigner transitivity judgments for the pantomimes and classifier constructions, with the bottom-up analysis providing a stronger, more convincing result. For lexical verbs, only the top-down analysis was relevant and it performed weakly, providing little explanatory power. When interpreting these results, we look to the semantics of the stimuli to explain the observed differences between classes: pantomimes and classifier constructions both encode events of motion and manipulation (by human hands), the transitivity of which may be encoded using a limited set of strategies. By contrast, lexical verbs denote a multitude of event types, with properties of those events (and not necessarily their transitivity) being preferentially encoded compared to the encoding of transitivity. That is, the resolution of transitivity is a much more difficult problem when looking at lexical verbs. </div><div><br></div><div>This dissertation contributes to the growing body of literature that appreciates how linguistic and paralinguistic forms may be both (para)linguistic and iconic at the same time. It further helps disentangle at least two different types of iconicities (lexical vs. structural), which may be selectively active in some signs or constructions xvi but not others. We also argue from our results that pantomimes are not holistic units, but instead combine elements of form and meaning in an analogous way to classifier constructions. Finally, this work also contributes to the discussion of how Language could have evolved in the species from a gesture-first perspective: The ‘understanding’ of others’ object-directed (i.e. transitive) manual actions becomes communicative.</div>

No description available.

This study attempts to shed light on the efficacy of crowdsourcing as a means of translating web content in Canada. Within, we seek to explore and understand if a model can be created that can estimate the effectiveness of crowdsourced translation as a means of bridging the Canadian Linguistic Digital Divide. To test our hypotheses and models, we use structural equation modeling techniques coupled with confidence intervals for comparing experimental crowdsourced translation to both professional and machine translation baselines. Furthermore, we explore a variety of factors which influence the quality of the experimental translations, how those translations performed in the context of their source text, and the ways in which the views of the quality of the experimental translations were measured before and after participants were made aware of how the experimental translations were created.

This study attempts to shed light on the efficacy of crowdsourcing as a means of translating web content in Canada. Within, we seek to explore and understand if a model can be created that can estimate the effectiveness of crowdsourced translation as a means of bridging the Canadian Linguistic Digital Divide. To test our hypotheses and models, we use structural equation modeling techniques coupled with confidence intervals for comparing experimental crowdsourced translation to both professional and machine translation baselines. Furthermore, we explore a variety of factors which influence the quality of the experimental translations, how those translations performed in the context of their source text, and the ways in which the views of the quality of the experimental translations were measured before and after participants were made aware of how the experimental translations were created.