Crowdsourcing is the outsourcing of a job or task to a large group of individuals. Crowdsourcing has emerged from the concepts of Outsourcing, Open Source Software (OSS) Collaboration, Open Innovation, and User Innovation. While Crowdsourcing has provided an innovative way in which work can be outsourced to a large group of people, the advent of Mobile Telephony in Africa has provided a whole new dimension. This is the merging of the concepts of Crowdsourcing and Mobile Telephony, to form Mobile Crowdsourcing. Mobile Crowdsourcing has the potential to contribute significantly to the use of Information Technology (IT) in developing countries by providing a platform that would enable people such as those in peri-urban Kenyan communities, to utilise their mobile handsets to perform a set of mobile-based tasks. Payment for these tasks is made possible through mobile money platforms such as “M-Pesa”. Such innovation could provide a means for social empowerment for many of these unemployed technology users. This study sets forth to examine a set of factors that are likely to predict the “participation intention” of peri-urban Kenyan youths in Mobile Crowdsourcing. Motivational Theory, and the Theory of Reasoned Action (TRA) form the core of the theoretical framework used for this study. The McKnight Model is used as a supporting theory, to examine “trusting beliefs”. In addition, the constructs “perceived credibility”, “social influence” and “community identification” are derived from prior studies that use Socio Cognitive Theory and an extended version of the Technology Acceptance Model (TAM). These also play a supporting role. Using a survey instrument, data was collected from peri-urban youths in four peri-urban communities, and 279 usable responses were obtained for this study. Findings show that “self-development”, “integrity”, and “reputation” are the most significant predictors of “participation intention”. These three variables account for 17% of the variance in “participation intention”. Contrary to suggestions made in prior literature on Crowdsourcing, “monetary compensation” was not found to be a key motivator. This finding will no doubt spark future debate as to the role money plays in Crowdsourcing, especially in Africa. Additional findings show that “attitude” was found to be a strong mediator of the relationship between “technology anxiety” and “participation intention”. Moreover, “community identification” was found to be a full moderator of the relationship between “social influence” and “participation intention”. Findings made uncovered new insights about the perceptions and attitudes of mobile phone users in developing countries. Contributions made to theory and practice are also discussed.

The purpose of this study is to explore crowdsourcing as a research paradigm for creating evidence-based practice research in the field of speech pathology. Using an Internet survey, respondents provided de-identified information about one patient with aphasia they had treated in the previous year. The respondents were then asked to rate the success of treatment. Analysis and grading of the responses was performed to identify which responses were usable for the purpose of planning a treatment for a patient with similar demographics and diagnostic make-up. Results showed that crowdsourcing is a viable research method; however, further refinements to the collection and analysis are required before it can be an effectively used. / text

In this thesis we investigate two main problems: 1) inferring consensus from disparate inputs to improve quality of crowd contributed data; and 2) developing a reliable crowd-aided IR evaluation framework. With regard to the first contribution, while many statistical label aggregation methods have been proposed, little comparative benchmarking has occurred in the community making it difficult to determine the state-of-the-art in consensus or to quantify novelty and progress, leaving modern systems to adopt simple control strategies. To aid the progress of statistical consensus and make state-of-the-art methods accessible, we develop a benchmarking framework in SQUARE, an open source shared task framework including benchmark datasets, defined tasks, standard metrics, and reference implementations with empirical results for several popular methods. Through the development of SQUARE we propose a crowd simulation model that emulates real crowd environments to enable rapid and reliable experimentation of collaborative methods with different crowd contributions. We apply the findings of the benchmark to develop reliable crowd contributed test collections for IR evaluation. As our second contribution, we describe a collaborative model for distributing relevance judging tasks between trusted assessors and crowd judges. Based on prior work's hypothesis of judging disagreements on borderline documents, we train a logistic regression model to predict assessor disagreement, prioritizing judging tasks by expected disagreement. Judgments are generated from different crowd models and intelligently aggregated. Given a priority queue, a judging budget, and a ratio for expert vs. crowd judging costs, critical judging tasks are assigned to trusted assessors with the crowd supplying remaining judgments. Results on two TREC datasets show significant judging burden can be confidently shifted to the crowd, achieving high rank correlation and often at lower cost vs. exclusive use of trusted assessors. / text

Magíster en Gestión para la Globalización / Se propone diseñar y especificar una propuesta de valor innovadora para los clientes en el mercado de las poleras estampadas, en conjunto con un plan de negocios para la creación de una empresa en este mercado basado en el crowdsourcing de obras de arte. El concepto de crowdsourcing consiste en externalizar tareas a cargo de un grupo numeroso de personas a través de una convocatoria abierta. El desarrollo de internet y la digitalización han abierto la puerta para acceder, sin límite de fronteras, al experto exacto que se necesita para un problema particular. En este proyecto, se plantea utilizar esta nueva herramienta para crear un concepto nuevo en el mercado de las poleras estampadas, que ha estado basado en la homogenización y estandarización de ofertas. Se plantea aportar valor al cliente utilizando artistas y diseñadores de clase mundial, en poleras exclusivas y de calidad superior, comercializando estos productos como verdaderas obras de arte. Para esto se plantea el uso del Modelo Canvas ideal para la creación de modelos de negocios basados en la creación, captura y entrega de valor. El Modelo Canvas define nueve elementos, los cuales se potencian entre si para generar propuestas de valor diferentes e innovadoras, exactamente lo que se necesita para este proyecto. Se provee en este documento una propuesta de valor potente, no solamente hacia los usuarios y potenciales clientes, sino también para los artistas proveedores de las obras a reproducir. De esta propuesta, derivará un plan de negocios completo incluyendo la especificación del modelo canvas y procesos operacionales. Finalmente se expone en este documento la viabilidad del proyecto y su potencial éxito basado en los análisis de Flujo de caja y análisis de sensibilidad. El escenario luce favorable para este negocio con un Valor Presente Neto de $91,317US y una TIR de 29% considerando una inversión inicial de $19,242US principalmente en maquinaria. Lo anterior considerando un periodo de evaluación de 6 Años y una tasa de descuento de un 12.3%. Con estos antecedentes se aprecia la relevancia del factor precio de venta que tiene el mayor impacto sobre el resultado del proyecto. Como recomendaciones, es posible mencionar un mayor análisis sobre el precio de venta, siguiendo el último comentario y su relevancia en el impacto del proyecto, mejores estimaciones ayudarían a definir mucho mejor el valor adecuado de venta. Adicionalmente, los datos sugieren que la asociación de los productos a elementos de arte tienen un impacto directo en la disposición a pagar por los mismos, aumentando considerablemente. Este factor podría potenciar este u otros proyectos utilizando un concepto similar.

Object recognition and manipulation are critical for enabling robots to operate within a household environment. There are many grasp planners that can estimate grasps based on object shape, but these approaches often perform poorly because they miss key information about non-visual object characteristics, such as weight distribution, fragility of materials, and usability characteristics. Object model databases can account for this information, but existing methods for constructing 3D object recognition databases are time and resource intensive, often requiring specialized equipment, and are therefore difficult to apply to robots in the field. We present an easy-to-use system for constructing object models for 3D object recognition and manipulation made possible by advances in web robotics. The database consists of point clouds generated using a novel iterative point cloud registration algorithm, which includes the encoding of manipulation data and usability characteristics. The system requires no additional equipment other than the robot itself, and non-expert users can demonstrate grasps through an intuitive web interface with virtually no training required. We validate the system with data collected from both a crowdsourcing user study and a set of grasps demonstrated by an expert user. We show that the crowdsourced grasps can produce successful autonomous grasps, and furthermore the demonstration approach outperforms purely vision-based grasp planning approaches for a wide variety of object classes.

With the development and proliferation of new social and connective technologies, crowdsourcing is becoming a viable method for conducting many types of work. At the same time, however, these developments are progressing more quickly than the law and raising new legal questions that often do not have definite answers yet. This thesis address some of these legal issues that crowdsourcing raises. In this thesis, we begin by addressing four areas that might lead to legal problems in the near future. First, we look at the labor and employment law issues that might arise from online crowdlabor markets like Amazon Mechanical Turk (www.mturk.com) and oDesk (www.odesk.com). Then we discuss inventorship issues under patent law that services like InnoCentive might experience. Next, we consider how data security laws could be problematic for open innovation projects like the Netflix challenge. Finally, we explore potential intellectual property ownership problems under copyright law. After discussing these topics, this thesis then turns to examine in detail the area of crowdfunding. As the name suggests, crowdfunding refers to process of raising money through crowdsourcing. Until recently, one type of crowdfunding known as crowdfinance was largely illegal under federal securities laws. However, the law in this area is starting to change. In this chapter, we look at four different models for crowdfunding: donation, lending, reward/prepurchase, and equity investment. Following that, we consider how federal securities regulation might apply to crowdfunding, particularly the equity investment model. Finally we conduct a content analysis of three legislative proposals to create a limited exemption for crowdfunding in securities law that the U.S. Congress recently considered. Finally, we assess how crowdsourcing platforms use private contracts to bind their users to certain terms and conditions. This chapter begins with a primer on contract law. Then we examine the enforceability of standardized online agreements. Following that, we review several provisions that are common to nearly all crowdsourcing platforms. Finally, we conduct a content analysis of the specific Terms of Use contracts of several crowdsourcing platforms. / text

Ontology learning supports ontology engineers in the complex task of creating an ontology. Updating ontologies at regular intervals greatly increases the need for expensive expert contribution. This naturally leads to endeavors to automate the process wherever applicable. This paper presents a model for automated ontology learning and a prototype which demonstrates the feasibility of the proposed approach in learning lightweight domain ontologies. The system learns ontologies from heterogeneous sources periodically and delegates all evaluation processes, eg. the verification of new concept candidates, to a crowdsourcing framework which currently relies on Games with a Purpose. Furthermore, we sketch ontology evolution experiments to trace trends and patterns facilitated by the system.(authors' abstract)

Crowdsourcing systems - like Ushahidi (for crisis mapping), Foldit (for protein folding) and Duolingo (for foreign language learning and translation) - have shown the effectiveness of intelligently organizing large numbers of people to solve traditionally vexing problems. Unfortunately, new crowdsourcing platforms are emerging to support the coordinated dissemination of spam, misinformation, and propaganda. These “crowdturfing” systems are a sinister counterpart to the enormous positive opportunities of crowdsourcing; they combine the organizational capabilities of crowdsourcing with the ability to widely spread artificial grass root support (so called “astroturfing”). This thesis begins a study of crowdturfing that targets social media and proposes a framework for “pulling back the curtain” on crowdturfers to reveal their underlying ecosystem. Concretely, this thesis (i) analyzes the types of campaigns hosted on multiple crowdsourcing sites; (ii) links campaigns and their workers on crowdsourcing sites to social media; (iii) analyzes the relationship structure connecting these workers, their profile, activity, and linguistic characteristics, in comparison with a random sample of regular social media users; and (iv) proposes and develops statistical user models to automatically identify crowdturfers in social media. Since many crowdturfing campaigns are hidden, it is important to understand the potential of learning models from known campaigns to detect these unknown campaigns. Our experimental results show that the statistical user models built can predict crowdturfers with very high accuracy.

With the wide popularity of smart devices such as smartphones, smartwatches, and smart cameras, Mobile Crowdsensing (MCS) and Crowdsourcing (CS) have been broadly applied for collecting data from a large group of ordinary participants. The quality of participants' contributed data, however, is hard to guarantee, and as such it is critical to develop efficient and effective methods to automatically improve data quality over MCS/CS platforms. In this thesis, we propose three machine learning-based solutions for data quality enhancement in different participatory MCS/CS scenarios. Our solutions aim at the data extraction phase as well as the data collection phase of participatory MCS/CS, including (1) trustworthy information extraction from conflicting data, (2) recognition of learning patterns, and (3) worker recruitment based on interactive training and learning pattern extraction. The first one is designed for the data extraction phase and the other two for the data collection phase. First, to derive reliable data from diverse or even conflicting labels from the crowd, we design a mechanism to infuse knowledge from domain experts into the labels from the crowd to automatically make correct decisions on classification-based MCS tasks. Our solution, named EFusion, utilizes a probabilistic graphical model and the expectation maximization (EM) algorithm to infer the most likely expertise level of each crowd worker, the difficulty level of tasks, and the ground truth answers. Furthermore, we introduce a method to extend EFusion from solving binary classification problems to handling multi-class classification problems. We evaluate EFusion using real-world case studies as well as simulations. Evaluation results demonstrate that EFusion can return more accurate and stable classification results than the majority voting method and state-of-the-art methods. Second, we propose Goldilocks, an interactive learning pattern recognition framework that can identify suitable participants whose performance follows desired learning patterns. To accurately extract a participant's learning pattern, we first estimate the impact of previous training questions on the participant before she answers a new question. After the participant answers each new question, we adjust the estimation of her capability by considering a quantitative measure of the impact of previous questions and her answer to the new question. Based on the extracted learning curve of each participant, we recruit the candidates, who have showed good learning capability and desired learning patterns, for the formal MCS/CS task. We further develop a web service over Amazon Web Services (AWS) that automatically adjusts questions to maximize individual participants' learning performance. This website also profiles the participants' learning patterns, which can be used for task assignment in MCS/CS. Third, we present HybrTraining, a hybrid deep learning framework that captures each candidate’s capability from a long-term perspective and excludes the undesired candidates in the early stage of the training phase. Using two collaborative deep learning networks, HybrTraining can dynamically match participants and MCS/CS tasks. In detail, we build a deep Q-network (DQN) to match the candidates and training batches in the training phase, and develop a long short-term memory (LSTM) model that extracts the learning patterns of different candidates and helps the DQN make better worker-task matching decisions. We build HyberTraining on Compute Canada and evaluate it over two scientific datasets. For each dataset, the learning data of candidates is collected with a Python-based Django website over Amazon Elastic Compute Cloud (Amazon EC2). Evaluation results show that HybrTraining can increase data collection efficiency and improve data quality in MCS/CS. / Graduate / 2022-08-19

Historians, like many types of scholars, are often researchers and educators, and both roles involve significant interaction with primary sources. Primary sources are not only direct evidence for historical arguments but also important materials for teaching historical thinking skills to students in classrooms, and engaging the broader public. However, finding high quality primary sources that are relevant to a historian's specialized topics of interest remains a significant challenge. Automated approaches to text analysis struggle to provide relevant results for these "long tail" searches with long semantic distances from the source material. Consequently, historians are often frustrated at spending so much time on manually the relevance of the contents of these archives other than writing and analysis. To overcome these challenges, my dissertation explores the use of crowdsourcing to support historians in analysis of primary sources. In four studies, I first proposed a class-sourcing model where historians outsource historical analysis to students as a teaching method and students learn historical thinking and gain authentic research experience while doing these analysis tasks. Incite, a realization of this model, deployed in 15 classrooms with positive feedback. Second, I expanded the class-sourcing model to a broader audience, novice (paid) crowds and developedthe Read-agree-predict (RAP) technique to accurately evaluate relevance between primary sources and research topics. Third, I presented a set of design principles for crowdsourcing complex historical documents via the American Soldier project on Zooniverse. Finally, I developed CrowdSCIM to help crowds learn historical thinking and evaluated the tradeoffs between quality, learning and efficiency. The outcomes of the studies provide systems, techniques and design guidelines to 1) support historians in their research and teaching practices, 2) help crowd workers learn historical thinking and 3) suggest implications for the design of future crowdsourcing systems. / Ph. D. / Historians, like many types of scholars, are often researchers and educators, and both roles involve significant interaction with primary sources. Primary sources are not only direct evidence for historical arguments but also important materials for teaching historical thinking skills to students in classrooms, and engaging the broader public. However, finding highquality primary sources that are relevant to a historian’s specialized topics of interest remains a significant challenge. Automated approaches to text analysis struggle to provide relevant results for these “long tail” searches with long semantic distances from the source material. Consequently, historians are often frustrated at spending so much time on manually the relevance of the contents of these archives other than writing and analysis. To overcome these challenges, my dissertation explores the use of crowdsourcing to support historians in analysis of primary sources. In four studies, I first proposed a class-sourcing model where historians outsource historical analysis to students as a teaching method and students learn historical thinking and gain authentic research experience while doing these analysis tasks. Incite, a realization of this model, deployed in 15 classrooms with positive feedback. Second, I expanded the class-sourcing model to a broader audience, novice (paid) crowds and developed the Read-agree-predict (RAP) technique to accurately evaluate relevance between primary sources and research topics. Third, I presented a set of design principles for crowdsourcing complex historical documents via the American Soldier project on Zooniverse. Finally, I developed CrowdSCIM to help crowds learn historical thinking and evaluated the tradeoffs between quality, learning and efficiency. The outcomes of the studies provide systems, techniques and design guidelines to 1) support historians in their research and teaching practices, 2) help crowd workers learn historical thinking and 3) suggest implications for the design of future crowdsourcing systems.