The ubiquity of digital media provides an unprecedented possibility to redefine the process and methods of design. Through experience from a series of creative projects, I investigate how certain aspects of the web allow for design to attain a degree of autonomy, thus producing results that go beyond anticipation and expertise of the author. Utilizing an unlimited capacity to store the content and retrieve immediate feedback, the designer’s role can be shifted to that of an initiator defining rules and boundaries, from which the process can evolve independently based on the input of users and data. The design output in such conditions is the development of schemes in which the author remains, but is marginalized as a producer – consciously restraining his level of control. Rather then struggling to add something singular to the bottomless pit of information, the designer is relegated in the role of medium: collecting and recombining the bits of fragmented data into the structures conveying meaning.

The aim of this thesis is to develop a gamified crowdsourcing platform for transcription and metadata tagging of digitised text documents, and further to examine whether the platform can stimulate engagement in archives and be used as an educational resource. The case, around which the thesis project is conducted, is the governmental Archives of Bosnia and Herzegovina and its historical documents which were damaged in a fire in 2014.  Theories about engagement, archival pedagogy, and gamification guided the development of the platform. Bosnian-Herzegovinian school children, History teachers, and staff of the Archives of Bosnia and Herzegovina tested the platform and shared their thoughts of it in focus group discussions and interviews. The developed platform consists of a game interface, a database, and an interface for results viewing and searching. It is web-based, and coded in HTML, CSS, JavaScript, and PHP. The platform stimulated engagement among the school children who tested it. They expressed interest in reading and learning more about archival documents, in long-term preservation of information, and in the archive’s mission and future. Judging from the discussions and interviews, the platform could well serve as an educational resource. The History teachers showed interest in using the platform in their teaching, and found support for it in the curriculum for the History subject. A gamified crowdsourcing platform has been developed, and the perception among school children, History teachers, and archive staff was overall very positive. Crowdsourcing is not just about sourcing out tasks to the crowd. Co-creation of content could furthermore stimulate learning and engagement, and thereby tie a bound between the public and the archive. Gamification can successfully make the overarching goal of crowdsourcing understandable and concrete for the platform users. / Uppsatsen syftar till att utveckla en spelifierad crowdsourcing-plattform för transkription och metadatataggning av digitaliserade textdokument samt att undersöka huruvida plattformen kan väcka engagemang för arkiv och användas som en utbildningsresurs. Det fall runt vilket uppsatsprojektet kretsar är Bosnien-Hercegovinas centrala arkivmyndighet och dess historiska dokument som skadades i en brand 2014. Teorier om engagemang, arkivpedagogik och gamification vägledde utvecklingen av plattformen. Skolbarn och historielärare i Bosnien-Hercegovina samt personal på arkivmyndigheten provade plattformen och delade med sig av sina tankar om den i fokusgruppsdiskussioner och intervjuer. Den utvecklade plattformen består av ett spelgränssnitt, en databas och ett gränssnitt för resultatvisning och sökning. Den är webbaserad och skriven i HTML, CSS, JavaScript och PHP. Plattformen väckte engagemang hos de barn som provade den. De uttryckte intresse av att läsa och lära sig mer om arkivdokument, av långsiktigt bevarande samt av arkivets uppdrag och framtid. Plattformen skulle, av fokusgruppsdiskussionerna och intervjuerna att döma, kunna användas och fungera väl som utbildningsresurs. Historielärarna visade intresse av att använda plattformen i sin undervisning och fann stöd för det i historieämnets läroplan. En spelifierad crowdsourcing-plattform har utvecklats och mottagandet av den bland skolbarn, historielärare och arkivpersonal var överlag mycket positivt. Crowdsourcing handlar inte enbart om att outsourca arbetsuppgifter till massan. Medskapande kan dessutom stimulera lärande och engagemang och därigenom knyta band mellan allmänheten och arkivet. Spelifiering kan med framgång göra crowdsourcingens övergripande mål förståeligt och konkret för dem som använder plattformen.

Le crowdsourcing est une technique qui permet de recueillir une large quantité de données d'une manière rapide et peu onéreuse. Néanmoins, La disparité comportementale et de performances des "workers" d’une part et la variété en termes de contenu et de présentation des tâches par ailleurs influent considérablement sur la qualité des contributions recueillies. Par conséquent, garder leur légitimité impose aux plateformes de crowdsourcing de se doter de mécanismes permettant l’obtention de réponses fiables et de qualité dans un délai et avec un budget optimisé. Dans cette thèse, nous proposons CAWS (Context AwareWorker Selection), une méthode de contrôle de la qualité des contributions dans le crowdsourcing visant à optimiser le délai de réponse et le coût des campagnes. CAWS se compose de deux phases, une phase d’apprentissage opérant hors-ligne et pendant laquelle les tâches de l’historique sont regroupées de manière homogène sous forme de clusters. Pour chaque cluster, un profil type optimisant la qualité des réponses aux tâches le composant, est inféré ; la seconde phase permet à l’arrivée d’une nouvelle tâche de sélectionner les meilleurs workers connectés pour y répondre. Il s’agit des workers dont le profil présente une forte similarité avec le profil type du cluster de tâches, duquel la tâche nouvellement créée est la plus proche. La seconde contribution de la thèse est de proposer un jeu de données, appelé CrowdED (Crowdsourcing Evaluation Dataset), ayant les propriétés requises pour, d’une part, tester les performances de CAWS et les comparer aux méthodes concurrentes et d’autre part, pour tester et comparer l’impact des différentes méthodes de catégorisation des tâches de l’historique (c-à-d, la méthode de vectorisation et l’algorithme de clustering utilisé) sur la qualité du résultat, tout en utilisant un jeu de tâches unique (obtenu par échantillonnage), respectant les contraintes budgétaires et gardant les propriétés de validité en terme de dimension. En outre, CrowdED rend possible la comparaison de méthodes de contrôle de qualité quelle que soient leurs catégories, du fait du respect d’un cahier des charges lors de sa constitution. Les résultats de l’évaluation de CAWS en utilisant CrowdED comparés aux méthodes concurrentes basées sur la sélection de workers, donnent des résultats meilleurs, surtout en cas de contraintes temporelles et budgétaires fortes. Les expérimentations réalisées avec un historique structuré en catégories donnent des résultats comparables à des jeux de données où les taches sont volontairement regroupées de manière homogène. La dernière contribution de la thèse est un outil appelé CREX (CReate Enrich eXtend) dont le rôle est de permettre la création, l’extension ou l’enrichissement de jeux de données destinés à tester des méthodes de crowdsourcing. Il propose des modules extensibles de vectorisation, de clusterisation et d’échantillonnages et permet une génération automatique d’une campagne de crowdsourcing. / Crowdsourcing has proved its ability to address large scale data collection tasks at a low cost and in a short time. However, due to the dependence on unknown workers, the quality of the crowdsourcing process is questionable and must be controlled. Indeed, maintaining the efficiency of crowdsourcing requires the time and cost overhead related to this quality control to stay low. Current quality control techniques suffer from high time and budget overheads and from their dependency on prior knowledge about individual workers. In this thesis, we address these limitation by proposing the CAWS (Context-Aware Worker Selection) method which operates in two phases: in an offline phase, the correlations between the worker declarative profiles and the task types are learned. Then, in an online phase, the learned profile models are used to select the most reliable online workers for the incoming tasks depending on their types. Using declarative profiles helps eliminate any probing process, which reduces the time and the budget while maintaining the crowdsourcing quality. In order to evaluate CAWS, we introduce an information-rich dataset called CrowdED (Crowdsourcing Evaluation Dataset). The generation of CrowdED relies on a constrained sampling approach that allows to produce a dataset which respects the requester budget and type constraints. Through its generality and richness, CrowdED helps also in plugging the benchmarking gap present in the crowdsourcing community. Using CrowdED, we evaluate the performance of CAWS in terms of the quality, the time and the budget gain. Results shows that automatic grouping is able to achieve a learning quality similar to job-based grouping, and that CAWS is able to outperform the state-of-the-art profile-based worker selection when it comes to quality, especially when strong budget ant time constraints exist. Finally, we propose CREX (CReate Enrich eXtend) which provides the tools to select and sample input tasks and to automatically generate custom crowdsourcing campaign sites in order to extend and enrich CrowdED.

The issue that this study addresses is the public transportations difficulty to adapt and keep up with the continuously digitizing society. To address this problem, the purpose of this study will be to investigate what could be a contribution to the further improvements within public transportation systems. The study is established alongside with the recently started “Welcome onboard” project, where the purpose is to develop and extend the public transportation as we know it today. The aim is to through five different functional areas urge the utility, where crowdsourcing will play a central role. We are distinguishing three public transport authorities attitudes towards the different functions of the project. We will investigate the attitudes towards the Welcome  onboardprojects functions and if they could be a contribution to the RKM-companies future development. Furthermore, this thesis will also investigate how the RKM-companies would grade the different advantages of different functional areas, and in a extend what would bevital for an future implementation. The empirical data collection consist of interviews of informants with a deep understanding ofthe development of the public transportations. First we interviewed two key persons of the Welcome onboard project, to get a deep understanding of the project. Later on we interviewed key persons at Värmlandstrafiken and Västtrafik, that are two different regional public transportation authorities. To strengthen our results we also chose to interview Samtrafiken, which task are to develop collaborations and offer services within information and ticket solutions for the public transport industry. This is made in order to benefit both traffic the RKM and the travelers.  All the interviews were transcribed and analyzed, later the empirical data collection could be compared through different cases. These cases were categorized by information about the company that the interviewees where representing and the answers regarding the attitudes towards the “Welcome onboard”-project and its functions. The result from the thesis will give an understanding of Västtrafik, Värmlandstrafiken and Samtrafikens perspective of the functions and attitudes towards the further development of public transportation.

Modern traffic management systems often rely on static technologies, such as sensors and CCTV-cameras, in the gathering of data regarding the current traffic situation. Recent reports have shown that this method can result in a lack of coverage in Stockholm, Sweden. In addressing this issue, an alternative strategy to installing more sensors and CCTV-cameras could be to utilize crowdsourced traffic data from other sources, such as Waze. In order to examine the usage and potential utility of crowdsourced data in traffic management, the Swedish Transport Administration’s center in Stockholm, Trafik Stockholm, developed a web application which visualizes traffic data from both official sources and Waze. While the application was successful in doing so, it revealed the problem of integrating the traffic data from these two sources, as a significant portion of the data was redundant, and the reliability occasionally was questionable. This study aims at determining how issues regarding redundancy and reliability can be resolved in the integration of crowdsourced and official traffic data. Conducted using a design science research strategy, the study investigates these issues by designing and developing an artifact that implements integration methods to match alerts from the data sources based on temporal and spatial proximity constraints. The artifact was evaluated through test sessions in which real-time traffic data from all over Sweden was processed, and through acceptance testing with the stakeholders of the application. Analysis of the results from the evaluations shows that the artifact is effective in reducing the redundancy in the crowdsourced data and that it can provide a more solid ground for reliability assessment. Furthermore, the artifact met its expectations and requirements, demonstrating a proof-of-concept and a proof-of-acceptance. Based on these results, the study concludes that by analyzing temporal and spatial factors in crowdsourced data, redundancy issues in the integration of crowdsourced and official traffic can be resolved to a large extent. Furthermore, it is concluded that reliability issues in the same context can be resolved to a high degree by managing redundancy factors in combination with general traffic management factors. While the study is focused on traffic management, the issues of redundancy and reliability are not restricted to crowdsourced data in this context specifically. Thus, the results of the study are potentially of interest to researchers investigating other areas of application for crowdsourcing as well.

Submitted by Silvana Teresinha Dornelles Studzinski (sstudzinski) on 2015-10-06T11:55:43Z No. of bitstreams: 1 Tiago Sommer_.pdf: 3876808 bytes, checksum: fe1d7120f07a0ab16d7f187824a760b6 (MD5) / Made available in DSpace on 2015-10-06T11:55:43Z (GMT). No. of bitstreams: 1 Tiago Sommer_.pdf: 3876808 bytes, checksum: fe1d7120f07a0ab16d7f187824a760b6 (MD5) Previous issue date: 2015-04-10 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Congestionamentos são um dos principais problemas nas cidades grandes. Eles geram poluição, stress emocional aos motoristas e desgaste das vias, o que demanda maior gasto para mantê-las em bom estado. Além disso, a ampliação dessas (para melhorar o fluxo) nem sempre é possível. Dessa forma, é preciso buscar outras alternativas. Uma possível solução é incentivar o uso do transporte público provendo informações de qualidade aos usuários e, para isso, existem os Sistemas de Informação ao Usuário (SIU). Estes sistemas computacionais visam prover melhores informações sobre o transporte coletivo para os usuários, atraindo-os para o uso do transporte público em detrimento aos veículos privados. Um dos grandes problemas de um SIU é manter seus dados atualizados, uma vez que as informações do transporte público se alteram periodicamente. Se o SIU não se atualiza constantemente, os usuários podem receber informações desatualizadas, podendo assim, abandonar o sistema. A gestão e atualização centralizada dos dados por órgãos gestores nem sempre se mostra suficiente e eficiente. Para resolver este problema, é proposto neste trabalho um sistema unificado de crowdsourcing e gamificação para SIU. O crowdsourcing visa permitir que os próprios usuários colaborem com informações para atualizar o sistema. Porém, o emprego exclusivo de crowdsourcing pode não ser suficiente, pois o sistema depende da participação ativa dos usuários para se manter atualizado. Se os usuários não estiverem motivados a compartilhar, o sistema se desatualizará e deixará de ser útil. Desse modo, a gamificação é utilizada neste trabalho para motivar os usuários a utilizarem mais o sistema e também a compartilharem com mais informações. Quanto maior o número de usuários interessados no sistema, maior as chances de se conseguir informações. Da mesma forma, quanto mais informações, mais confiável será o sistema e, consequentemente, mais usuários estarão interessados em utilizá-lo. O trabalho proposto traz uma contribuição significativa para a área de Sistemas Inteligentes de Transporte, pois não há conhecimento de trabalhos que integrem gamificação e crowdsourcing em um SIU para motivação da participação dos usuários com compartilhamento de informações. Para verificar se o trabalho atingiu os seus objetivos, foi realizada uma avaliação qualitativa com três usuários. Esses três usuários foram observados in loco durante a utilização do sistema em paradas na cidade de Porto Alegre. Foi igualmente realizada uma outra avaliação experimental quantitativa em que 23 usuários assistiram um vídeo sobre o sistema e responderam um questionário com questões fechadas. A análise das respostas dos usuários mostrou indícios que eles se sentem motivados em contribuir com o transporte público. Além disso, os usuários também reportaram estar interessados na gamificação do sistema. / Traffic jams are one of the main problems in the cities. The expansion of urban roads is not a sustainable solution, for this reason other alternatives are necessary. One possible solution are the User Information Systems (UIS). These computer systems aim to provide information about the public transport for users, attracting them to use public transport instead of private vehicles. The main problem of an UIS is to keep its data updated. To solve this problem, we propose an unified model of crowdsourcing and gamification for an UIS. The crowdsourcing aims to allow users to collaborate with updated data. However, only the crowdsourcing may not be enough, because the system will depend on the users participation to keep itself updated. If the users are not motivated to cooperate, the system will downgrade and will no longer be useful. Thus, gamification will be used to motivate users to use the system and also to collaborate with more data. The greater the number of users interested in the system, the better are the chances of getting useful data from users. Likewise, with more information, the system will be more reliable and therefore more users will be interested in using it. The proposed work contains a significative contribution to the Intelligent Transportation Systems area because we are not aware of other studies that integrate gamification and crowdsourcing in an UIS. To verify if the work achieved its objectives, a qualitative evaluation was performed with three users. The users were observed in situ while using the system at bus stops in the city of Porto Alegre. A quantitative experiment evaluation was also conducted in which 23 users watched a video about the system and answered questionnaires with closed questions. The analysis of the users responses showed evidence that they feel motivated to contribute to public transportation. In addition, users also reported to be interested in the gamification of the system.

Submitted by Filipe dos Santos (fsantos@pucsp.br) on 2017-12-06T11:00:19Z No. of bitstreams: 1 Cora Rodrigues Junqueira.pdf: 1152321 bytes, checksum: fad8c04ec3b1bfaf39d87f4cdfe98153 (MD5) / Made available in DSpace on 2017-12-06T11:00:19Z (GMT). No. of bitstreams: 1 Cora Rodrigues Junqueira.pdf: 1152321 bytes, checksum: fad8c04ec3b1bfaf39d87f4cdfe98153 (MD5) Previous issue date: 2017-11-11 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / The present work introduces a bibliographical research with critical reflection on the impact of the collaborative processes in the network, specifically in mobile applications, in the smart cities. The study was based on the contextualization of the theme, with a brief description of the concept of smart cities and origin of the term, as well as the survey and definition of concepts related to the processes of collective intelligence and its derivations, and concepts related to big data and networked organizations, based on the work of Clay Shirky, Manuel Castells, Lucia Santaella, James Surowiecki and Henry Jenkins. After the support of the historical survey, this work approaches the case studies of two companies engaged as mobilizing agents, focused on one of the basic principles of improvement of the city, main focus of this work: urban mobility, with the common ground on data update from a collaborative system, collective intelligence and big data. In order to investigate how and to what extent the collective power is applied in smart cities, acting both a collector of urban data and a agent of change, it was necessary, in short, to understand the functioning of the smart city; to detect how the constitution of communities takes place; the influence of ICT on the development of the city; to analyze how people engage, from the participatory culture and the formation of communities in the applications; in conclusion, to understand the convergence between big data and crowdsourcing in the smart city / O presente trabalho apresenta uma pesquisa de caráter bibliográfico, com reflexão crítica referente ao impacto dos processos colaborativos em rede, especificamente em aplicativos móveis, nas cidades inteligentes. O estudo se deu a partir da contextualização do tema, com uma breve descrição do conceito de cidades inteligentes e origem do termo, para assim entendêla na atualidade, bem como o levantamento e definição de conceitos relativos aos processos de inteligência coletiva e suas derivações, assim como os conceitos relativos ao big data e às organizações em rede, baseando-se nos trabalhos de, principalmente, Clay Shirky, Manuel Castells, Lucia Santaella, James Surowiecki e Henry Jenkins. Após o respaldo do levantamento histórico, chega-se nos estudos de casos de duas empresas engajadas como agentes mobilizadores, voltadas para um dos princípios básicos de aprimoramento da cidade, ponto focal deste trabalho: mobilidade urbana, tendo como ponto em comum a atualização de dados a partir de um sistema colaborativo, partindo da inteligência coletiva e big data. Afim de investigar como e em até que ponto o poder do coletivo é aplicado em cidades inteligentes, atuando tanto como coletor de dados urbanos, quanto agente de mudanças, foi necessário, em suma, compreender o funcionamento da cidade inteligente; detectar como se dá a constituição de comunidades; a influência das TIC no desenvolvimento da cidade; analisar como se dá o engajamento das pessoas, a partir da cultura participativa e da formação de comunidades nos aplicativos; em suma, entender a convergência entre big data e crowdsourcing na cidade inteligente

Le repliement de biomolécules à partir de méthodes computationelles reste un grand défi. Plus particulièrement, les simulations de dynamique moléculaire tout-atomes sont intrinsèquement longues et ne permettent pas encore d’atteindre l’échelle de temps de la microseconde de façon courante. En général, un approche gros-grain est préférée pour simuler des systèmes plus grands et des échelles de temps plus longues. Les approches automatiques comme la dynamique moléculaire ne tiennent pas compte de l’expertise de l’investigateur. Ce travail de thèse explore le repliement des biomolécules au moyen de simulations de dynamique moléculaire interactives avec les modèles gros-grains OPEP et HiRE-RNA, respectivement dédiés aux acides aminés et nucléiques. Les simulations interactives sont comme les simulations classiques, mais permettent en plus à l’utilisateur d’appliquer des forces sur une sélection d’atomes et d’observer la réaction du système en direct pendant que la simulation tourne depuis un logiciel de visualisation moléculaire. Des développements logiciels dédiés ont été faits dans un de ces programmes, UnityMol, couplé aux simulations gros-grain OPEP et HiRE-RNA. Ce travail est complété par une incursion dans la biologie intégrative. L’utilisation de modèles théoriques et expérimentaux est proposée sous deux formes: l’introduction de biais dans les simulations pour les faire converger plus rapidement vers des résultats plausibles et le guidage des utilisateurs au cours de sessions interactives. Cette réflexion montre la complémentarité des méthodes théoriques et des méthodes expérimentales pour l’étude des biomolécules. Quelques essais de repliement ont été menés par des simulations interactives avec nos outils. Une approche dite collaborative (ou plus généralement “crowdsourcing”) au repliement de molécules d’ARN gros-grains avec le modèle HiRE-RNA fut menée. Le repliement de peptides a suivi dans une configuration de laboratoire avec OPEP. En complément, des aspects de réalité virtuelle et des améliorations de performance du logiciel de simulation de réseaux de ressorts BioSpring ont été explorés / The folding of biomolecules by computational methods remains a big challenge. Most notably, all-atom molecular dynamics (MD) simulations are intrinsically time consuming and do not yet commonly reach the microsecond time scale. Generally, a coarse-grained approach is preferred to simulate bigger systems and larger time scales. Automated approaches like MD do not account for the investigator expertise. The present thesis explores the folding of biomolecules with interactive molecular dynamics (IMD) simulations using the OPEP and HiRE-RNA models, respectively for amino acids and nucleic acids. IMD is like MD, but in addition, the user can apply forces on a selection of atoms and see the reaction of the system live from a molecular visualization software while the simulation is running. Dedicated software developments were done in such a program named UnityMol, coupled with coarse-grained OPEP and HiRE-RNA simulations. The picture is completed with an incursion into integrative biology. The use of theoretical and experimental models is proposed in two declinations: biasing MD simulations to faster converge to plausible results and guide users during interactive sessions. This work shows the complementarity of experimental and theoretical methods when it comes to biomolecules. A few trials at folding with IMD and our set of tools are exposed: mainly a crowdsourcing approach to RNA folding with coarse-grained HiRE-RNA models and the interactive folding of peptides in a laboratory setup of OPEP simulations. In complement, virtual reality aspects and performance enhancement of a spring network model simulation package named BioSpring have been explored

This research study is an analysis of the intertwining concepts of collaboration, prosumption, and communication in the contemporary digital educational space. The major objective of this thesis is to understand the operationalisation of the collaborative production of free and open artefacts. For this purpose, a framework is established for investigating communication and interaction activities of the people involved and their usage of ICT supported communications. Three exemplary projects are then examined using this framework: Schulbuch-O-Mat, a pilot project in which an electronic OER schoolbook is created; Memrise, an online platform which offers a wide range of user-created learning courses in a variety of topics; and VroniPlag Wiki, a wiki in which dissertations are collaboratively examined in terms of plagiarism. Data for this study were collected in the form of a comparative multiple-case study using a mix of methods such as interviews, questionnaires, participant observation, and the analysis of documents. Using situational analysis, a comparative analysis was conducted in terms of the three main aspects of this study: prosumption, collaboration, and communications. The results show that a balanced mix of communication enabling tools and services facilitates high collaboration. It was also concluded that collaborative working procedures benefit from diverse ways to address participants. The obtained data support the view that less hierarchical structures simplify collaborations and prosumption. Further research should address the issue of digital labour and exploitation as well as the management of conflicts in prosumer orientated collaborations. As a further outcome, a system of categories was derived in the analysis part of this thesis. This categorisation system serves as a foundation towards an approach of analysing collaborative communication.

The Internet has evolved into a platform on which large numbers of individuals take action and join in collaborations via crowdsourcing, social media, and electronic commerce. When designing social and economic systems on the Internet, a key challenge is understanding how to promote particular desired behaviors and outcomes. I call this problem computational environment design. Notable abilities afforded by the Internet, such as the ability to recruit large numbers of individuals to join problem-solving efforts via crowdsourcing and social media, and the ability to engage in a data-driven iterative design process, are creating new opportunities and inspiring new methods for computational environment design. This dissertation focuses on these abilities and proposes an approach for arriving at effective designs by reasoning and learning about characteristics of participants and how these characteristics interact with a system’s design to influence behavior. The dissertation consists of two major components. The first component focuses on designing crowdsourcing and human computation systems that leverage a crowd to solve complex problems that require effective coordination among participants or the recruitment of individuals with relevant expertise. I show how reasoning about crowd abilities and limitations can lead to designs that make crowdsourcing complex tasks feasible, effective, and efficient. The solutions introduce new design patterns and methods for human computation and crowdsourcing; notable contributions include a crowdware design for tackling human computation tasks with global constraints, and incentive mechanisms for task routing that harness people’s expertise and social expertise by engaging them in both problem solving and routing. The second component focuses on understanding how to design effective environments automatically. I introduce a general active, indirect elicitation framework for automated environment design that learns relevant characteristics of participants based on observations of their behavior and optimizes designs based on learned models. Theoretical contributions include developing an active, indirect elicitation algorithm for a sequential decision-making setting that is guaranteed to discover effective designs after few interactions. Practical contributions include applications of the active, indirect elicitation framework to crowdsourcing. Specifically, I demonstrate how to automatically design tasks and synthesize workflows when optimizing for desired objectives given resource constraints. / Engineering and Applied Sciences