Diese Arbeit soll dafür einen Überblick der verschiedenen Crowdsourcing-Ansätze liefern und die Anforderungen und Probleme des Crowdsourcing untersuchen. Dafür wird zunächst das Zusammenführen unterschiedlicher Datenbestände (data matching) betrachtet. Darauf folgt eine allgemeine Vorstellung des Crowdsourcing, um abschließend die unterschiedlichen Ansätze untersuchen zu können.

Application that combines crowdsourcing and gamification elements should be usable by as many passengers as possible, considering all the different smartphones available on the market and people with disabilities. Thus, most popular platforms for cross-platform mobile application development should be compared in order to choose the appropriate one for this project. Finally, application prototype should be further tested to gather more feedback about design and concept for improvements and future studies.  Such solution is not meant to replace current methods of information gathering, but instead should be one of the available tools.

<p>When humans delegate tasks---whether to human workers or robots---they do so either to trade money for time, or to leverage additional knowledge and capabilities. For complex tasks, however, describing the work to be done requires substantial effort, which reduces the benefit to the requester who delegates tasks. On one hand, human workers---e.g., crowd workers, friends or colleagues on social network, factory workers---have diverse knowledge and level of commitment, making it difficult to achieve joint efforts towards the requester's goal. In contrast, robots and machines have clearly defined capabilities and full commitment, but the requester lacks an efficient way to coordinate them for flexible workflows. </p><p> </p><p> This dissertation presents a series of workflows and systems to enable efficient work transfer to human workers or robots. First, I present BlueSky, a system that can automatically coordinate hundreds of crowd workers to enumerate ideas for a given topic. The latent structure of the idea enumeration task is decomposed into a three-step workflow to guide the crowd workers. Second, I present CoStory, a system that requests alternative designs from friends or colleagues by decomposing the design task into hierarchical chunks. Third, I present AdapTutAR, a system that delegates machine operation tasks to workers through adaptive Augmented Reality tutorials. Finally, I present Vipo, a system that allows requesters to customize tasks for robots and smart machines through spatial-visual programming. This dissertation demonstrates that decomposing latent task structure enables task delegation in an on-demand, scalable, and distributed way.</p>

Ethical, cheap, and scalable, purposeful games leverage player entertainment to incentivise contributors in language resourcing. However, discourse is scarce around the enjoyability of these games, whose playerbases are divided between a tiny minority of reliable contributors and a vast majority of inconsistent contributors. This study aims to deepen the discourse around design possibilities tailored to the unevenly contributing playerbases of such games by building on player-reported data to create three engaging personas and narrative scenarios. Using Pruitt and Grudin’s way of weighing feature suitability in persona-focused design, social incentives and majority voting are indicated as the most and least prominent features, respectively. Indeed, the weight of the primary persona, representing 3.5% of the playerbase, is 72%, more than double the combined weight, 56%, of the remaining 96.5% of the playerbase. Sticking to the original definition of purposeful games is essential for any gaming approach to crowdsourced data collection to remain ethical, cheap, and scalable.

Most engineering curricula in the United States include some form of major design project experiences for students, such as capstone courses or design-build-fly projects. Such courses are examples of project-based learning (PBL). Part of PBL is to prepare students—and future engineers—to deal with and prevent common project failures such as missing requirements, overspending, and schedule delays. <i>But how well are students performing once they join the workforce?</i> Unfortunately, despite our best efforts to prepare future engineers as best we can, the frequency of failures of complex projects shows no signs of decreasing. In 2020 only 53% of projects were on time, 59% within budget, and 69% met their goal, as reported by the Project Management Institute. If we want to improve success rates in industry projects, letting students get the most out of their PBL experience and be better prepared to deal with project failures before they join the workforce may be a viable starting point. <br><br>The overarching goal of this dissertation is to identify and suggest improvements to areas that PBL lacks when it comes to preparing students for failure, to investigate student behaviors that lead to project failures, and to improve these behaviors by providing helpful feedback to students. <br><br>To investigate the actions and behaviors that lead to events that cause failures in student projects, I introduced “crowd signals”, which are data collected directly from the students that are part of a project team. In total, I developed 49 survey questions that collect these crowd signals. To complete the first part of the dissertation, I conducted a first experiment with 28 student teams and their instructors in two aerospace engineering PBL courses at Purdue University. The student teams were working on aircraft designs or low-gravity experiments.<br><br><i>Does PBL provide sufficient opportunities for students to fail safely, and learn from the experience? How can we improve?</i> To identify areas that PBL may lack, I compared industry failure cause occurrence rates with similar rates from student teams in PBL courses, and then provided recommendations to PBL instructors. Failure causes refer to events that frequently preceded budget, schedule, or requirements failures in industry, and are identified from the literature. Through this analysis, I found that PBL does not prepare students sufficiently for situations where the failure cause missing a design aspect occurs. The failure cause is fundamentally linked to proper systems engineering: it represents a scenario where, for example, students failed to consider an important requirement during system development, or did not detect a design flaw, or component incompatibility. I provided four recommendations to instructors who want to give their students more opportunities to learn from this failure cause, so they are better prepared to tackle it as engineers. <br><br><i>Is crowdsourced information from project team members a good indicator of future failure occurrences in student projects?</i> I developed models that predict the occurrence of future budget, schedule, or requirements failures, using crowd signals and other information as inputs, and interpreted those models to get an insight on which student actions are likely to lead to project failures. The final models correctly predict, on average, 73.11±6.92% of budget outcomes, 75.27%±9.21% of schedule outcomes, and 76.71±6.90% of technical requirements outcomes. The previous status of the project is the only input variable that appeared to be important in all three final predictive models for all three metrics. Overall, crowdsourced information is a useful source of knowledge to assess likelihood of future failures in student projects. <br><br><i>Does targeted feedback that addresses the failure causes help reduce failures in student projects?</i> To improve student behaviors that lead to project failures, I used correlations between failure measures and the crowd signals as a guide to generate 35 feedback statements. To evaluate whether the feedback statements help reduce project failures in the student teams, I conducted a second experiment at Purdue University with 14 student teams and their instructors. The student teams were enrolled in aircraft design, satellite design, or propulsion DBT courses. The student teams were split in two treatment groups: teams that received targeted feedback (i.e., feedback that aimed to address the failure causes that the specific team is most prone to) and teams that received non-targeted feedback (i.e., feedback that is positive, but does not necessarily address the failure causes the specific team is most prone to). Through my analysis, I found that my targeted feedback does not reduce the failure occurrences in terms of any metrics, compared to the non-targeted feedback. However, qualitative evaluations from the students indicated that student teams who received targeted feedback made more changes to their behaviors and thought the feedback was more helpful, compared to the student teams who received non-targeted feedback.<br><br>

Crowdsourcing has become a popular choice for tackling problems that neither computers nor humans alone can solve with adequate speed, cost, and quality. However, instructing crowds to execute tasks in the manner expected by the requesters is challenging. It depends on not only requesters’ task design abilities but also workers’ understanding of the tasks. Task design bridges the communication gap between workers and requesters, which consists of instructions, payment, time limit on task, and the interface for workers to work on. It remains an underdeveloped but important topic that needs further exploration for improving crowdsourcing experience. My research studies task delivery from requesters to crowd workers. The goal is to improve the communication between the two and, in turn, increase accuracy of results and decrease variability due to differing interpretations and perspectives. Specifically, this dissertation presents a series of studies to show that high-quality results can be obtained from human workers through improved task design, by 1) designing incentives to recruit workers with the appropriate skills for given tasks, 2) designing unambiguous instructions to clearly express task requirements, 3) choosing the correct strategy to communicate the requisite task knowledge with workers, and 4) enhancing requesters’ ability to rapidly prototype Augmented Reality (AR) instructions. This dissertation demonstrates that crowdsourcing quality is improved when the tasks are communicated using mediums and structures that align with workers’ preference and utility

The aim of this dissertation is to study two problems of distributed nearest neighbor classification (DiNN) systematically. The first one compares two DiNN classifiers based on different schemes: majority voting and weighted voting. The second one is an extension of the DiNN method to the crowdsourcing application, which allows each worker data has a different size and noisy labels due to low worker quality. Both statistical guarantees and numerical comparisons are studied in depth.<br><div><br></div><div><div>The first part of the dissertation focuses on the distributed nearest neighbor classification in big data. The sheer volume and spatial/temporal disparity of big data may prohibit centrally processing and storing the data. This has imposed a considerable hurdle for nearest neighbor predictions since the entire training data must be memorized. One effective way to overcome this issue is the distributed learning framework. Through majority voting, the distributed nearest neighbor classifier achieves the same rate of convergence as its oracle version in terms of the regret, up to a multiplicative constant that depends solely on the data dimension. The multiplicative difference can be eliminated by replacing majority voting with the weighted voting scheme. In addition, we provide sharp theoretical upper bounds of the number of subsamples in order for the distributed nearest neighbor classifier to reach the optimal convergence rate. It is interesting to note that the weighted voting scheme allows a larger number of subsamples than the majority voting one.</div></div><div><br></div><div>The second part of the dissertation extends the DiNN methods to the application in crowdsourcing. The noisy labels in crowdsourcing data and different sizes of worker data will deteriorate the performance of DiNN methods. We propose an enhanced nearest neighbor classifier (ENN) to overcome this issue. Our proposed method achieves the same regret as its oracle version on the expert data with the same size. We also propose two algorithms to estimate the worker quality if it is unknown in practice. One method constructs the estimators for worker quality based on the denoised worker labels through applying kNN classifier on expert data. Unlike previous worker quality estimation methods, which have no statistical guarantee, it achieves the same regret as the ENN with observed worker quality. The other method estimates the worker quality iteratively based on ENN, and it works well without expert data required by most previous methods.<br></div>

<div>Face redaction is used to deidentify images of people. Most approaches depend on face detection, but automated algorithms are still not adequate for sensitive applications in which even one unredacted face could lead to irreversible harm. Human annotators can potentially provide the most accurate detection, but only trusted annotators should be allowed to see the faces of privacy-sensitive applications. Redacting more images than trusted annotators could accommodate requires a new approach. </div><div>This dissertation leverages the characteristics of human perception of faces in median-filtered images in a human computation algorithm to engage crowd workers to redact faces—without revealing the identities. IntoFocus, a system I developed, permits robust face redaction with probabilistic privacy guarantees. The system's design builds on an experiment that measured the filter levels and conditions where participants could detect and identify faces. </div><div> Pterodactyl is a system that focuses on increasing the productivity of crowd-based face redaction systems. It uses the AdaptiveFocus filter, a filter that combines human perception of faces in median filtered images with a convolutional neural network to estimate a median filter level for each region of the image to allow the faces to be detected and prevent them from being identified.</div>

The aim of the thesis is to describe and evaluate the general volunteering opportunities on the internet and describe volunteer experience in the operation, development and changes of the Open Directory Project (ODP) as an example of content management project. This thesis seeks to collectively map the issue capture and volunteering in the Internet environment. It specifies the sources and monitors the possibilities to participate in volunteer activities on the Internet.

This study explores the social and cultural implications of photography and metadata crowdsourcing at the level of smaller communities as a means of preserving, rekindling, and representing local culture. A multiple case study approach is used, each case focusing on various instances of photography crowdsourcing both in and outside the digital environment, in order to quantitatively analyse a hand-picked selection of photographic material, its respective metadata, and the social ecosystem. The study finds that by circulating photographic material within their communities of provenance, the information crowdsourced differs from institutional metadata standards, as communities evaluate the material through its socially embeddedness, its ability to be evocative of a collective memory, and as an active means of reclaiming identities. Crowdsourcing activities of the sort are an alternative means of record making, not bound by issues of copyright or ownership. By taking into account the unique experiences and identities of non-centric communities, the gap between GLAMs and non-affiliated projects can enrich national cultural heritage and possibly reevaluate the importance of the photograph as a socially salient and relevant medium.