Staging a crowdsourcing challenge includes opening up your organization to the outside. When doing so, some organizations choose to collaborate with an existing platform owner. This may establish new relationships and opportunities for learning. This research aims to investigate the set of learnings and adjustments that occur between the seekers and platform owners in the challenge process. To do this, a qualitative study in the form of semi-structured interviews, with five seekers, two platform owners, and two specialists in the field, was conducted. Our findings indicate various learnings and adjustments for seekers and platform owners. Seekers learn and adjust by defining the challenge properly, becoming more independent in running challenges, changing their internal culture, and accepting new intellectual property insights. Platform owners learn and adjust by listening carefully to understand the seekers’ goals and by implementing transparency in their way of working. We conclude that through implementing these learnings and adjustments, seekers and platform owners collaboratively gain more confidence and become better at running challenges.

The purpose of this study is to explore if monetary reward negatively affects people’s willingness to participate in crowdsourcing projects and to see how the relationship between monetary rewards and the framing of the crowdsourcing problem affects participation in crowdsourcing contests. A two-phase data collection method is used to answer these questions; a survey to identify the people who had participated in crowdsourcing projects and a focus group with the suitable candidates to discuss the research questions further. According to our findings, framing a crowdsourcing project as a good cause is not a strong enough motivation to convince people to participate in a challenge. People usually look for a benefit (financial or personal) in a challenge when deciding to participate. On the other hand, offering a reward for a crowdsourcing contest that is held for a good cause increases people’s willingness to participate. Potential participants react differently to a reward that is larger than usual. While more experienced participants feel extra motivated by large rewards, those who have less experience in crowdsourcing projects are more likely to see the large reward as a threat, decreasing their chances of winning, thus, reducing their willingness to participate in those challenges.

(in English): The thesis deals with online labour markets. They are differentiated against job portals and have their characteristics and specific variations relevant to workers and employers analysed. The thesis also differentiates a Talent Market. The analysis identifies geographical distribution of workers as well as the workforce composition related to primary area of expertise. A significant borderline between the dominant fields of work in the east and west is found. The analysis also focuses on fees imposed by individual portals and their relation to the utility of a given portal. In the conclusion, recommendations concerning choosing the most suitable market with regard to the nature and length of the project are given.

I ett ständigt åldrande samhälle har Sveriges kommuner ideligen utvecklat äldreomsorgen genom välfärdsteknologin, och i synnerhet inom IKT. Denna studie handlar om hur man designar en mockup prototyp till en applikation som riktar sig till att hjälpa äldre människor. Tanken med applikationen "I’m here" är att vara ett digitalt verktyg inom den svenska äldreomsorgen: hemtjänsten - där äldre personer kan använda den för att annonsera/fråga om hjälp. Genom en process av crowdsourcing kan registrerade arbetstagare svara på förfrågningarna och hitta den äldre personen genom sin position på kartan. Genom ett Design Science- tillvägagångssätt gjordes en första mockup prototyp enligt teorin om Pervasive Information Architecture, och evaluerades av 18 äldre deltagare mellan 68-86 år. Deltagarnas design-önskemål implementerades i den andra mockup prototypen. Resultaten av evalueringen kan anses som grundläggande och allmän fast de är kontextuella och hör till en mockup prototyp av en specifik applikation, kan den producerade kunskapen betraktas som grundläggande och allmän. Denna studie bidrar också med idén om en potentiell applikation "I’m here". Ytterligare forskning är nödvändig för att genomföra större studier om design för äldre personer. Detta skulle också bidra till en fullständig utveckling av applikationen "I’m here". / In an ever aging society, Sweden’s municipalities have steadily developed elderly health care through welfare technologies, and within ICT in particular. This study is about how to design a mockup prototype to a mobile application that is targeted towards helping elderly people. The idea of the application ‘I’m here’ is to be a digital tool within the Swedish elderly care ‘hemtjänsten’, where elderly persons are able to use it to announce a request for help. Through a process of crowdsourcing, registered workers are able to answer the requests and find the elderly person by his/her position on the map. Through a Design Science approach, an initial mockup prototype was made according to the theory of Pervasive Information Architecture, and was evaluated by 18 elderly participants between the ages of 68-86. Their design evaluations were considered and implemented in a second mockup prototype. The results are contextual and belong to a mockup prototype of a specific application, the produced knowledge can be considered foundational and general. This study also contributes to the idea of a potential application ‘I’m here’. Further research is need to conduct larger studies on design for elderly persons. Also, this would contribute to the full development of the application ‘I’m here’.

Collaboration has always been critical to the success of new product development teams, and the advent of geographically dispersed teams has significantly altered the way that team members interact. Multi-user computer-aided design (MUCAD) and crowdsourcing are two results of efforts to enable collaboration between geographically dispersed individuals. In this research, a study was done to investigate the differences in performance between MUCAD and single-user CAD teams, in which teams competed to create the best model of a hand drill. This was done across a three-day period to recreate the scenario found in industry. It was found that MUCAD increases awareness of teammates' activities and increases communication between team members. Different sources of frustration for single-user and multi-user teams were identified, as well as differing patterns of modeling style. These findings demonstrate that MUCAD software has significant potential to improve team collaboration and performance. A second study explored a number of potentially significant factors in MUCAD team performance, including leadership, design style, unfamiliar parts, knowledge transfer, individual experience, and team composition. In this study, teams of undergraduate mechanical engineering students worked together to complete tasks using NXConnect, a MUCAD plugin for NX developed at Brigham Young University. A primary finding was that having an appointed leader for a MUCAD team improves performance, in particular when that leader works with the team in creating the CAD model. It was also found that creating a framework to aid in organizing and coordinating the creation of the CAD model may decrease the time required for completion. In the final study, the possibility of using crowdsourcing to complete complex product design tasks was explored. In this study, a process for crowdsourcing complex product design tasks was developed, as well as a website to act as the platform for testing this process. A crowd consisting of engineering and technology students then worked together on the website to design a frisbee tracking device. The crowd was able to collaborate to accomplish some detailed product design tasks, but was not able to develop a complete product. Major findings include the need for more formal leadership and crowd organization, the need for better decision making mechanisms, and the need for a better model for engaging crowd members on a consistent basis. It was also found that crowd members had a greater willingness to pay for the product they developed than individuals who had not worked on the project. Results also show that although crowd members were often frustrated with the collaboration process, they enjoyed being able to work with a large group of people on a complex project.

Collaboration has always been critical to the success of new product development teams, and the advent of geographically dispersed teams has significantly altered the way that team members interact. Multi-user computer-aided design (MUCAD) and crowdsourcing are two results of efforts to enable collaboration between geographically dispersed individuals. In this research, a study was done to investigate the differences in performance between MUCAD and single-user CAD teams, in which teams competed to create the best model of a hand drill. This was done across a three-day period to recreate the scenario found in industry. It was found that MUCAD increases awareness of teammates' activities and increases communication between team members. Different sources of frustration for single-user and multi-user teams were identified, as well as differing patterns of modeling style. These findings demonstrate that MUCAD software has significant potential to improve team collaboration and performance. A second study explored a number of potentially significant factors in MUCAD team performance, including leadership, design style, unfamiliar parts, knowledge transfer, individual experience, and team composition. In this study, teams of undergraduate mechanical engineering students worked together to complete tasks using NXConnect, a MUCAD plugin for NX developed at Brigham Young University. A primary finding was that having an appointed leader for a MUCAD team improves performance, in particular when that leader works with the team in creating the CAD model. It was also found that creating a framework to aid in organizing and coordinating the creation of the CAD model may decrease the time required for completion. In the final study, the possibility of using crowdsourcing to complete complex product design tasks was explored. In this study, a process for crowdsourcing complex product design tasks was developed, as well as a website to act as the platform for testing this process. A crowd consisting of engineering and technology students then worked together on the website to design a frisbee tracking device. The crowd was able to collaborate to accomplish some detailed product design tasks, but was not able to develop a complete product. Major findings include the need for more formal leadership and crowd organization, the need for better decision making mechanisms, and the need for a better model for engaging crowd members on a consistent basis. It was also found that crowd members had a greater willingness to pay for the product they developed than individuals who had not worked on the project. Results also show that although crowd members were often frustrated with the collaboration process, they enjoyed being able to work with a large group of people on a complex project.

Nowadays, employees have to work with applications, technical services, and systems every day for hours. Hence, performance degradation of such systems might be perceived negatively by the employees, increase frustration, and might also have a negative effect on their productivity. The assessment of the application's performance in order to provide a smooth operation of the application is part of the application management. Within this process it is not sufficient to assess the system performance solely on technical performance parameters, e.g., response or loading times. These values have to be set into relation to the perceived performance quality on the user's side - the quality of experience (QoE). This dissertation focuses on the monitoring and estimation of the QoE of enterprise applications. As building models to estimate the QoE requires quality ratings from the users as ground truth, one part of this work addresses methods to collect such ratings. Besides the evaluation of approaches to improve the quality of results of tasks and studies completed on crowdsourcing platforms, a general concept for monitoring and estimating QoE in enterprise environments is presented. Here, relevant design dimension of subjective studies are identified and their impact of the QoE is evaluated and discussed. By considering the findings, a methodology for collecting quality ratings from employees during their regular work is developed. The method is realized by implementing a tool to conduct short surveys and deployed in a cooperating company. As a foundation for learning QoE estimation models, this work investigates the relationship between user-provided ratings and technical performance parameters. This analysis is based on a data set collected in a user study in a cooperating company during a time span of 1.5 years. Finally, two QoE estimation models are introduced and their performance is evaluated. / Heutzutage sind Geschäftsanwendungen und technische Systeme aus dem Arbeitsalltag vieler Menschen nicht mehr wegzudenken. Kommt es bei diesen zu Performanzproblemen, wie etwa Verzögerungen im Netzwerk oder Überlast im Datenzentrum, kann sich dies negativ auf die Effizienz und Produktivität der Mitarbeiter auswirken. Daher ist es wichtig aus Sicht der Betreiber die Performanz der Anwendungen und Systeme zu überwachen. Hierbei ist es allerdings nicht ausreichend die Qualität lediglich anhand von technischen Performanzparametern wie Antwortzeiten zu beurteilen. Stattdessen sollten diese Werte in Relation zu der von den Mitarbeitern wahrgenommenen Performanz oder Quality of Experience (QoE) gesetzt werden. Diese Dissertation beschäftigt sich mit dem Monitoring und der Abschätzung der QoE von Geschäftsanwendungen. Neben der Präsentation eines generellen Konzepts zum Monitoring und der Abschätzung der QoE im Geschäftsumfeld, befasst sich die Arbeit mit Aspekten der Erfassung von Qualitätsbewertungen durch die Nutzer. Dies umfasst einerseits die Evaluation von Ansätzen zur Verbesserung der Qualität von Aufgaben- und Studienergebnissen auf Crowdsourcing-Plattformen. Andererseits werden relevante Dimensionen des Designs von Studien zur Untersuchung der QoE von Geschäftsanwendungen aufgezeigt und deren Einfluss auf die QoE diskutiert und evaluiert. Letztendlich wird eine Methodik zur Erfassung von Qualitätsbewertungen durch Mitarbeiter während ihrer regulären Arbeit vorgestellt, welche implementiert und in einem kooperierenden Unternehmen ausgerollt wurde. Als Grundlage der Entwicklung eines QoE Abschätzungsmodells, untersucht diese Arbeit den Zusammenhang zwischen Bewertungen durch die Nutzer und technischen Performanzparametern. Die Untersuchungen erfolgen auf einem Datensatz, welcher in einer Studie über 1.5 Jahre in einem kooperierenden Unternehmen gesammelt wurde. Des Weiteren werden zwei Methoden zur Abschätzung der QoE präsentiert und deren Performanz evaluiert.

With the proliferation of social media, gathering data has became cheaper and easier than before. However, this data can not be used for supervised machine learning without labels. Asking experts to annotate sufficient data for training is both expensive and time-consuming. Current techniques provide two solutions to reducing the cost and providing sufficient labels: crowdsourcing and active learning. Crowdsourcing, which outsources tasks to a distributed group of people, can be used to provide a large quantity of labels but controlling the quality of labels is hard. Active learning, which requires experts to annotate a subset of the most informative or uncertain data, is very sensitive to the annotation errors. Though these two techniques can be used independently of one another, by using them in combination they can complement each other’s weakness. In this thesis, I investigate the development of active learning Support Vector Machines (SVMs) and expand this model to sequential data. Then I discuss the weakness of combining active learning and crowdsourcing, since the active learning is very sensitive to low quality annotations which are unavoidable for labels collected from crowdsourcing. In this thesis, I propose three possible strategies, incremental relabeling, importance-weighted label prediction and active Bayesian Networks. The incremental relabeling strategy requires workers to devote more annotations to uncertain samples, compared to majority voting which allocates different samples the same number of labels. Importance-weighted label prediction employs an ensemble of classifiers to guide the label requests from a pool of unlabeled training data. An active learning version of Bayesian Networks is used to model the difficulty of samples and the expertise of workers simultaneously to evaluate the relative weight of workers’ labels during the active learning process. All three strategies apply different techniques with the same expectation – identifying the optimal solution for applying an active learning model with mixed label quality to iii crowdsourced data. However, the active Bayesian Networks model, which is the core element of this thesis, provides additional benefits by estimating the expertise of workers during the training phase. As an example application, I also demonstrate the utility of crowdsourcing for human activity recognition problems.

One of the most striking trends in individuals’ careers over the last decade has been the dramatic increase in the proportion of the labor force working beyond their employers’ physical boundaries because of the digital revolution in the gig economy. This trend has drawn much attention in the changing nature of work, workplace and careers. However, little empirical research has explored how and why individuals behave in the interface between online platforms and traditional organizations. In my dissertation, I explore these questions by studying medical professionals’ digital careers in the Chinese healthcare crowdsourcing industry, also known as “mobile doctors.” First, by analyzing approximately 240-hour observations and 43 interviews with Chinese physicians, I identify a key issue in this new career – time conflict between crowdsourcing and traditional work. The findings show that physicians respond to time conflict in a variety of ways, including time theft, an essential yet under-researched construct in the crowdsourcing literature which reflects the tension between traditional work and crowdsourcing. Second, by analyzing archival data of 4,034 doctors’ 3.1 million time records on a Chinese healthcare platform across half a year, I show that time theft for crowdsourcing is related to the traditional work context, including hospitals’ boundary control and offline crowd worker social groups. Finally, I further explore, via interview data, why such seemingly costly and deviant time theft is adopted by mobile doctors. The findings reveal that medical professionals assume the extra burden of working for crowdsourcing with the hope of answering unfulfilled occupational callings in traditional work and adding meaning to their work. Overall, these findings contribute to a better understanding of the shifting nature of work and careers in the digital economy by documenting and explaining mobile doctors’ participation in this new world of work.

No / The first recorded crowdsourcing activity was in 1714 [1], with intermittent public event occurrences up until the millennium when such activities become widespread, spanning multiple domains. Crowdsourcing, however, is relatively novel as a methodology within virtual environment studies, in archaeology, and within the heritage domains where this research is focused. The studies that are being conducted are few and far between in comparison to other areas. This paper aims to develop a recent concept in crowdsourcing work termed `crowd behaviour mining' [2] using virtual environments, and to develop a unique concept in crowdsourcing activities that can be applied beyond the case studies presented here and to other domains that involve human behaviour as independent variables. The case studies described here use data from experiments involving separate heritage projects and conducted during two Royal Society Summer Science Exhibitions, in 2012 and 2015 respectively. `Crowd Behaviour Mining' analysis demonstrated a capacity to inform research in respect of potential patterns and trends across space and time as well as preferences between demographic user groups and the influence of experimenters during the experiments.