This thesis is centered around the problem of attribute learning -- using the joint effort of humans and machines to describe objects, e.g., determining that a piece of music is "soothing," that the bird in an image "has a red beak", or that Ernest Hemingway is an "Nobel Prize winning author." In this thesis, we present new methods for solving the attribute-learning problem using the joint effort of the crowd and machines via human computation games. When creating a human computation system, typically two design objectives need to be simultaneously satisfied. The first objective is human-centric -- the task prescribed by the system must be intuitive, appealing and easy to accomplish for human workers. The second objective is task-centric -- the system must actually perform the task at hand. These two goals are often at odds with each other, especially in the casual game setting. This thesis shows that human computation games can accomplish both the human-centric and task-centric objectives, if we first design for humans, then devise machine learning algorithms to work around the limitations of human workers and complement their abilities in order to jointly accomplish the task of learning attributes. We demonstrate the effectiveness of our approach in three concrete problem settings: music tagging, bird image classification and noun phrase categorization. Contributions of this thesis include a framework for attribute learning, two new game mechanisms, experiments showing the effectiveness of the hybrid human and machine computation approach for learning attributes in vocabulary-rich settings and under the constraints of knowledge limitations, as well as deployed games played by tens of thousands of people, generating large datasets for machine learning.

The tags and descriptions entered by video owners in video sharing sites are typically inadequate for retrieval purposes, yet the majority of video search still uses this text. This problem is escalating due to the ease with which users can self-publish videos, generating masses that are poorly labelled and poorly described. This thesis investigates how users tag videos and whether video tagging games can solve this problem by generating useful sets of tags. A preliminary study investigated tags in two social video sharing sites, YouTube and Viddler. YouTube contained many irrelevant tags because the system does not encourage users to tag their videos and does not promote tags as useful. In contrast, using tags as the sole means of categorisation in Viddler motivated users to enter a higher proportion of relevant tags. Poor tags were found in both systems, however, highlighting the need to improve video tagging. In order to give users incentives to tag videos, the VideoTag project in this thesis developed two tagging games, Golden Tag and Top Tag, and one non-game tagging system, Simply Tag, and conducted two experiments with them. In the first experiment VideoTag was a portal to play video tagging games whereas in the second experiment it was a portal to curate collections of special interest videos. Users preferred to tag videos using games, generating tags that were relevant to the videos and that covered a range of tag types that were descriptive of the video content at a predominately specific, objective level. Users were motivated by interest in the content rather than by game elements, and content had an effect on the tag types used. In each experiment, users predominately tagged videos using objective language, with a tendency to use specific rather than basic tags. There was a significant difference between the types of tags entered in the games and in Simply Tag, with more basic, objective vocabulary entered into the games and more specific, objective language entered into the non-game system. Subjective tags were rare but were more frequent in Simply Tag. Gameplay also had an influence on the types of tags entered; Top Tag generated more basic tags and Golden Tag generated more specific and subjective tags. Users were not attracted to use VideoTag by the games alone. Game mechanics had little impact on motivations to use the system. VideoTag used YouTube videos, but could not upload the tags to YouTube and so users could see no benefit for the tags they entered, reducing participation. Specific interest content was more of a motivator for use than games or tagging and that this warrants further research. In the current game-saturated climate, gamification of a video tagging system may therefore be most successful for collections of videos that already have a committed user base.

Game with a purpose (GWAP) is a concept that aims to utilize the hours spent in the world playing video games by everyday people to yield valuable data. The main objective of this research is to prove the feasibility of using the concept of GWAP for the segmentation and labeling of massive amount of 3D human body scan data. The rationale behind using GWAP as a method for mesh segmentation and labeling is that the current methods use expensive, time consuming computational algorithms to accomplish this task. Furthermore, the computer algorithms are not as detailed and specific as what natural human ability can achieve in segmentation tasks. The method presented in this paper overcomes the shortcomings of computer algorithms by introducing the concept of GWAP for human model segmentation. The actual process of segmenting and labeling the mesh becomes a form of entertainment rather than a tedious process, from which segmentation data is produced as a bi-product. In addition, the natural capabilities of the human visual processing systems are harnessed to identify and label various parts of the 3D human body shape, which in turn gives more details and specificity in segmentation. The effectiveness of the proposed game play mechanism is proven by experiments conducted in this study.

Human Computation methods such as crowdsourcing and games with a purpose (GWAP) have each recently drawn considerable attention for their ability to synergize the strengths of people and technology to accomplish tasks that are challenging for either to do well alone. Despite this increased attention, much of this transformation has been focused on a few selected areas of information science. This thesis contributes to the field of human computation as it applies to areas of information science, particularly information retrieval (IR). We begin by discussing the merits and limitations of applying crowdsourcing and game-based approaches to information science. We then develop a framework that examines the value of using crowdsourcing and game mechanisms to each step of an IR model. We identify three areas of the IR model that our framework indicates are likely to benefit from the application of human computation methods: acronym identification and resolution, relevance assessment, and query formulation. We conduct experiments that employ human computation methods, evaluate the benefits of these methods and report our findings. We conclude that employing human computation methods such as crowdsourcing and games, can improve the accuracy of many tasks currently being done by machine methods alone. We demonstrate that the best results can be achieved when human computation methods augment computer-based IR processes, providing an extra level of skills, abilities, and knowledge that computers cannot easily replicate.

Ethical, cheap, and scalable, purposeful games leverage player entertainment to incentivise contributors in language resourcing. However, discourse is scarce around the enjoyability of these games, whose playerbases are divided between a tiny minority of reliable contributors and a vast majority of inconsistent contributors. This study aims to deepen the discourse around design possibilities tailored to the unevenly contributing playerbases of such games by building on player-reported data to create three engaging personas and narrative scenarios. Using Pruitt and Grudin’s way of weighing feature suitability in persona-focused design, social incentives and majority voting are indicated as the most and least prominent features, respectively. Indeed, the weight of the primary persona, representing 3.5% of the playerbase, is 72%, more than double the combined weight, 56%, of the remaining 96.5% of the playerbase. Sticking to the original definition of purposeful games is essential for any gaming approach to crowdsourced data collection to remain ethical, cheap, and scalable.

In this thesis, an improved framework is proposed for categorizing existing gamified systems. Related works and real world examples of gamification are discussed and some areas where insufficient research exists. In order to address the identified research problems, an experimental gamified system was designed and implemented for sharing articles related to different aspects of life. The participants of the study were recruited among the users of an existing Iranian lifestyle site for female users. Therefore, the gender of the users was considered in the design. A wide range of gamification elements were implemented in the system to test the effectiveness of specific design features and gamification elements’ parameters in increasing user motivation, for example, the contingency of rewards and the use of sound and animation in badges. A detailed questionnaire was used to answer the research questions. The results suggest specific combinations of gamification elements and their parameters that can be successfully applied by designers of social sites for similar audience.

Crowdsourcing has been used in the cultural heritage domain for a variety of tasks. One of them is generation of descriptive metadata for digital archives. Gamification offers citizens a more entertaining way to interact with digital collections and generate useful metadata as a side effect of gameplay. The rise of social gaming on Facebook in recent years opens new horizons for cultural heritage institutions to leverage the capabilities of social networking platforms and to gain immediate access to millions of potential contributors. In this work, we explore the integration of social networks with crowdsourcing games for generating archival metadata. We studied crowdsourcing, gamification and social dynamics from the perspective of cultural heritage and combine their features in a metadata game prototype on the Facebook platform. We tested our prototype and evaluate its results by analysing participation, contribution and player feedback. The two-week testing phase showed promising results in terms of user engagement and produced metadata: almost 3000 tags were added, 90% of which were valid dictionary terms. We conclude that deploying metadata games on social networking platforms is a feasible method for digital archives to harness human intelligence from large shared spaces.

No description available.

Computer hardware and algorithm design have seen significant progress over the years. It is also seen that there are several domains in which humans are more efficient than computers. For example in image recognition, image tagging, natural language understanding and processing, humans often find complicated algorithms quite easy to grasp. This thesis presents the different case studies to learn human mapping strategy to solve the mapping problem in the area of coarse-grained reconfigurable architectures (CGRAs). To achieve optimum level performance and consume less energy in CGRAs, place and route problem has always been a major concern. Making use of human characteristics can be helpful in problems as such, through pattern recognition and experience. Therefore to conduct the case studies a computer mapping game called UNTANGLED was analyzed as a medium to convey insights of human mapping strategies in a variety of architectures. The purpose of this research was to learn from humans so that we can come up with better algorithms to outperform the existing algorithms. We observed how human strategies vary as we present them with different architectures, different architectures with constraints, different visualization as well as how the quality of solution changes with experience. In this work all the case studies obtained from exploiting human strategies provide useful feedback that can improve upon existing algorithms. These insights can be adapted to find the best architectural solution for a particular domain and for future research directions for mapping onto mesh-and- stripe based CGRAs.