Many current test collections require the use of expert judgments during construction. The true label of each document is given by an expert assessor. However, the cost and effort associated with expert training and judging are typically quite high in the event where we have a high number of documents to judge. One way to address this issue is to have each document judged by multiple non-expert assessors at a lower expense. However, there are two key factors that can make this method difficult: the variability across assessors' judging abilities, and the aggregation of the noisy labels into one single consensus label. Much previous work has shown how to utilize this method to replace expert labels in the relevance evaluation. However, the effects of relevance judgment errors on the ranking system evaluation have been less explored. This thesis mainly investigates how to best evaluate information retrieval systems with noisy labels, where no ground-truth labels are provided, and where each document may receive multiple noisy labels. Based on our simulation results on two datasets, we find that conservative assessors that tend to label incoming documents as non-relevant are preferable. And there are two important factors affect the overall conservativeness of the consensus labels: the assessor's conservativeness and the relevance standard. This important observation essentially provides a guideline on what kind of consensus algorithms or assessors are needed in order to preserve the high correlation with expert labels in ranking system evaluation. Also, we systematically investigate how to find the consensus labels for those documents with equal confidence to be either relevant or non-relevant. We investigate a content-based consensus algorithm which links the noisy labels with document content. We compare it against the state-of-art consensus algorithms, and find that, depending on the document collection, this content-based approach may help or hurt the performance.

Abstract Title: HelpTube: A Study on the Motivation behind Crowdsourcing with STD & Motivation Functions for Volunteer Work as Explanatory Models. Level: Bachelor Thesis in Business Administration; C-level. Authors: Honey H. Shahraki & Linnea M. Pira Supervisor: Jonas Kågström Date: 2013- June Background: In recent years the trend and usage of social networking has increased significantly amongst individuals as well as businesses. YouTube is one example of the social environment in which the public has an opportunity to act as both consumer and supplier. Prior research in this field emphasizes the importance of understanding individuals’ incentive to share their knowledge. Thus, what are the underlying objectives behind individuals’ actions and willingness to contribute to the accumulated knowledge? Aim of the study: The aim of the study is to build on prior research in the field of Self-Determination Theory (Deci & Ryan, 2000) and Clary et als (1998) research on functional motivation for volunteering. Subsequently, we then approach these manners in a crowsourcing context by measuring why YouTube users want to contribute to cumulative knowledge through a social media network. Method: This study is based on a quantitative and deductive approach. Grounded on the indices of Self-Determination-Theory (Aspiration Index) and Functional Motivation Theory of Volunteering (Volunteer Function Inventory) we assembled six different categories. These constituted our online survey of a questionnaire containing 18 questions. The questionnaire was then designed in Google forms and sent out to YouTube users creating tutorials for various software programs. Further, the questionnaire responses were processed in the IV result and analysis chapter, where a statistical inquiry consisting of Factor analysis, Correlation analysis and Cluster analysis was made. Results & Analysis: The processing of the study’s result and analysis indicated that the YouTube-respondents were in line with both the Self-Determination Theory as well as Clary et al’s (1998) theory of the functional motivation for volunteering. The respondents valued each question in different degrees of esteem, for each set of categories that was formed in this study. The questions that the respondents tends to value the most were the following; to help people in need, to grow as a person and obtain different learning experiences, to have fun, to work towards the betterment of society, and to obtain new and different perspectives. The least valued questions for the respondents were the following; monetary rewards, to create tutorials because friends do it, and to escape bad feelings. Further, it can be concluded that the majority of questions in the categories, Personal Growth and Intrinsic Projection are valued most highly by the YouTube respondents. Conclusion: After the result and analysis process, a pattern was observed. The studied respondents exhibited an inclination towards internal motivators. Accordingly, we were able to indetify strong links between the categories; Instrinsic Projection, Personal Growth and Societal Values, and the respondents. This is consistent with the Self-Determination Theory as well as the Functional Motivation Theory for Volunteering. Additionally, we can comment that the studied group on YouTube also fulfills the criterias of the crowdsourcing concept. Future Research: In our opinion there is currently a knowledge gap that needs to be filled, in order to understand and utilize the phenomenon’s full capacity. Questions regarding the underlying motivation of individuals that participate, and engage their time and energy towards a further development of accumulated knowledge, is evidently scarce. Since this study primarily focuses on ascertaining the YouTube users' underlying motives, a proposal for further research can be to apply this model to a different type of network. An additional proposal could be for future researchers to develop or improve the study’s compiled model and thereby achieve more precise answers. Further, we hope that more scholars develop recognition of the Crowdsourcing phenomenon, hence attaining a greater knowledge and contributing organizations with practical tools that they can apply and make use of. Contribution: With the study’s gained knowledge about the participants underlying motivation factors, organizations using this phenomenon could increase the participation-levels and thereby extracting a more reliable source of information. This would help organizations to obtaining an additional channel to reach and enlist the help of consumers worldwide. Keywords: Self-Determination Theory, Functional Volunteer Motivation, Crowdsourcing, Motivation, YouTube, Tutorial.

No description available.

The following Bachelor’s thesis explores the different uses of crowdsourcing by private organisations and analyses them internally, in terms of strategic capabilities. The purpose of this Bachelor’s thesis is to show the reader the different internal strategic issues resulting from the use of crowdsourcing by private organisations. The authors focused namely on crowd creation, crowdfunding and crowd voting through three private organisations using one of these types respectively in their business processes. The qualitative research was conducted through a multiple case study design and through interviews for the primary data collection. The results from the research varied from case to case. Firstly, the Ricola case has shown that crowd creation can especially have impacts on its physical strategic capabilities. Secondly, La Biscuiterie Jeannette’s case has indicated that crowdfunding strongly impacts its financial strategic capabilities. Thirdly, the case of Schneider has enabled to highlight on the one hand the growing importance of crowd voting and on the other hand that crowd voting had no major impacts on its strategic capabilities, yet. Finally, this research intended to give inspiration to other researchers into the field of crowdsourcing and its three subtypes. Therefore, this thesis can be a basis for further researches in this field.

Crowdfunding is an online fundraising technique, where individuals or groups ask for contributions from a large number of people, i.e. “the crowd”. Crowdfunding first appeared in 2003 and has doubled its financial scope every year since 2011. Yet, very little is known about it, especially from a societal perspective. In this thesis I have attempted to understand how the organizations surrounding crowdfunding have structured themselves into an organizational filed, and also how “the crowd” might have granted this field special characteristics. This since “the crowd” is an element not earlier taken into consideration when studying field emergence. I have conducted a content analysis with data from 170 crowdfunding platforms and 190 media articles. I found that several events coincided in 2009 and opened up for the emergence of the crowdfunding field, and also that the understanding of crowdfunding’s purpose has shifted since 2003, from culture projects to start-ups. Regarding the special characteristic of the field it seems like the online nature enabled a fierce structuration pace, and also that “the crowd” constitutes an entity that cannot be captured by our traditional understanding of fields. The results suggest that when “the crowd” takes on functions in a field, the functions become invisible, and this has implications for future research regarding organizational fields.

Background: Organizational spending on marketing needs to be justified and therefore measured. New technology has enabled new ways of conducting market research. Research question: How can knowledge be transferred from a crowd of consumers to a company operating in the retail industry? Purpose: To explore the process of knowledge transfer in a new type of market research company by creating an understanding of (1) how to gather knowledge through engaging and motivating a crowd to share information, (2) how to analyze and transfer it to the clients, and in the end (3) how the clients receive the information and are able to create knowledge internally. Methodology: Qualitative single case study through semi-structured interviews with the case company and two of their clients. This was combined with secondary data and observations. Conclusions: Motivated users are important when gathering knowledge through crowdsourcing. Focusing on gathering and transferring explicit knowledge makes it more actionable and therefore more valuable when it comes to market insights. Externalization was found to only be partly possible in the case company, which strengthens established theories published after Nonaka and Takeuchi (1995). Absorptive capacity and relationships had influence on the knowledge transfer and how the results were acted upon in the client organizations.

Many current test collections require the use of expert judgments during construction. The true label of each document is given by an expert assessor. However, the cost and effort associated with expert training and judging are typically quite high in the event where we have a high number of documents to judge. One way to address this issue is to have each document judged by multiple non-expert assessors at a lower expense. However, there are two key factors that can make this method difficult: the variability across assessors' judging abilities, and the aggregation of the noisy labels into one single consensus label. Much previous work has shown how to utilize this method to replace expert labels in the relevance evaluation. However, the effects of relevance judgment errors on the ranking system evaluation have been less explored. This thesis mainly investigates how to best evaluate information retrieval systems with noisy labels, where no ground-truth labels are provided, and where each document may receive multiple noisy labels. Based on our simulation results on two datasets, we find that conservative assessors that tend to label incoming documents as non-relevant are preferable. And there are two important factors affect the overall conservativeness of the consensus labels: the assessor's conservativeness and the relevance standard. This important observation essentially provides a guideline on what kind of consensus algorithms or assessors are needed in order to preserve the high correlation with expert labels in ranking system evaluation. Also, we systematically investigate how to find the consensus labels for those documents with equal confidence to be either relevant or non-relevant. We investigate a content-based consensus algorithm which links the noisy labels with document content. We compare it against the state-of-art consensus algorithms, and find that, depending on the document collection, this content-based approach may help or hurt the performance.

No description available.

Mobile apps can make use of the rich data and sensors available on smartphones to offer compelling services. However, the use of sensitive resources by apps is not always justified, which has led to new kinds of privacy risks and challenges. While it is possible for app market owners and third-parties to analyze the privacy-related behaviors of apps, present approaches are difficult and tedious. I present two iterations of the design, implementation, and evaluation of a system, Gort, which enables more efficient app analysis, by reducing the burden of instrumenting apps, making it easier to find potential privacy problems, and presenting sensitive behavior in context. Gort interacts with apps while instrumenting them to detect sensitive information transmissions. It then presents this information along with the associated app context to a crowd of users to obtain their expectations and comfort regarding the privacy implications of using the app. Gort also runs a set of heuristics on the app to flag potential privacy problems. Finally, Gort synthesizes the information obtained through its analysis and presents it in an interactive GUI, built specifically for privacy analysts. This work offers three distinct new advances over the state of the art. First, Gort uses a set of heuristics, elicited through interviews with 12 experts, to identify potential app privacy problems. Gort heuristics present high-level privacy problems instead of the overwhelming amount of information offered through existing tools. Second, Gort automatically interacts with apps by discovering and interacting with UI elements while instrumenting app behavior. This eliminates the need for analysts to manually interact with apps or to script interactions. Third, Gort uses crowdsourcing in a novel way to determine whether app privacy leaks are legitimate and desirable and raises red flags about potentially suspicious app behavior. While existing tools can detect privacy leaks, they cannot determine whether the privacy leaks are beneficial or desirable to the user. Gort was evaluated through two separate user studies. The experiences from building Gort and the insights from the user studies guide the creation of future systems, especially systems intended for the inspection and analysis of software.

For decades, researchers have seen employee empowerment as the means to achieving a more committed workforce that would deliver better outcomes. The prior conceptual and descriptive research focused on structural empowerment, or workplace mechanisms for generating empowerment, and psychological empowerment, the felt empowerment. Responding to calls for intervention studies, this research experimentally tests the effects of structural empowerment changes, through different degrees of decision-making authority and access to customer-relationship information, on psychological empowerment and subsequent work-related outcomes. Using a virtual contact center simulation, crowdsourced workers responded to customer requests. Greater decision authority and access to customer-relationship information resulted in higher levels of psychological empowerment which in turn resulted in task satisfaction and task attractiveness outcomes among the crowdsourced customer service workers.