In the modern world, making informed decisions requires obtaining and aggregating relevant information about events of interest. For many political, business, and entertainment events, the information of interest only exists as opinions, beliefs, and judgments of dispersed individuals, and we can only get a complete picture by putting the separate pieces of information together. Thus, an important first step towards decision making is motivating the individuals to reveal their private information and coalescing the separate pieces of information together. In this dissertation, I study three information elicitation and aggregation methods, prediction markets, peer prediction mechanisms, and adaptive polling, using both theoretical and applied approaches. These methods mainly differ by their assumptions on the participants' behavior, namely whether the participants possess noisy or perfect information and whether they strategically decide on what information to reveal. The first two methods, prediction markets and peer prediction mechanisms, assume that the participants are strategic and have perfect information. Their primary goal is to use carefully designed monetary rewards to incentivize the participants to truthfully reveal their private information. As a result, my studies of these methods focus on understanding to what extent are these methods incentive compatible in theory and in practice. The last method, adaptive polling, assumes that the participants are not strategic and have noisy information. In this case, our goal is to accurately and efficiently estimate the latent ground truth given the noisy information, and we aim to evaluate whether this goal can be achieved by using this method experimentally. I make four main contributions in this dissertation. First, I theoretically analyze how the participants' knowledge of one another's private information affects their strategic behavior when trading in a prediction market with a finite number of participants. Each participant may trade multiple times in the market, and hence may have an incentive to withhold or misreport his information in order to mislead other participants and capitalize on their mistakes. When the participants' private information is unconditionally independent, we show that the participants reveal their information as late as possible at any equilibrium, which is arguably the worse outcome for the purpose of information aggregation. We also provide insights on the equilibria of such prediction markets when the participants' private information is both conditionally and unconditionally dependent given the outcome of the event. Second, I theoretically analyze the participants' strategic behavior in a prediction market when a participant has outside incentives to manipulate the market probability. The presence of such outside incentives would seem to damage the information aggregation in the market. Surprisingly, when the existence of such incentives is certain and common knowledge, we show that there exist separating equilibria where all the participants' private information is revealed and fully aggregated into the market probability. Although there also exist pooling equilibria with information loss, we prove that certain separating equilibria are more desirable than many pooling equilibria because the separating equilibria satisfy domination based belief refinements, maximize the social welfare of the setting, or maximize either participant's total expected payoff. When the existence of the outside incentives is uncertain, trust cannot be established and the separating equilibria no longer exist. Third, I experimentally investigate participants' behavior towards the peer prediction mechanisms, which were proposed to elicit information without observable ground truth. While peer prediction mechanisms promise to elicit truthful information by rewarding participants with carefully constructed payments, they also admit uninformative equilibria where coordinating participants provide no useful information. We conduct the first controlled online experiment of the Jurca and Faltings peer prediction mechanism, engaging the participants in a multiplayer, real-time and repeated game. Using a hidden Markov model to capture players' strategies from their actions, our results show that participants successfully coordinate on uninformative equilibria and the truthful equilibrium is not focal, even when some uninformative equilibria do not exist or result in lower payoffs. In contrast, most players are consistently truthful in the absence of peer prediction, suggesting that these mechanisms may be harmful when truthful reporting has similar cost to strategic behavior. Finally, I design and experimentally evaluate an adaptive polling method for aggregating small pieces of imprecise information together to produce an accurate estimate of a latent ground truth. In designing this method, we make two main contributions: (1) Our method aggregates the participants' noisy information by using a theoretical model to account for the noise in the participants' contributed information. (2) Our method uses an active learning inspired approach to adaptively choose the query for each participant. We apply this method to the problem of ranking a set of alternatives, each of which is characterized by a latent strength parameter. At each step, adaptive polling collects the result of a pairwise comparison, estimates the strength parameters from the pairwise comparison data, and adaptively chooses the next pairwise comparison question to maximize expected information gain. Our MTurk experiment shows that our adaptive polling method can effectively incorporate noisy information and improve the estimate accuracy over time. Compared to a baseline method, which chooses a random pairwise comparison question at each step, our adaptive method can generate more accurate estimates with less cost. / Engineering and Applied Sciences

The purpose of this study is to investigate a new interrogation tactic named Mockingbird. The tactic is derived from a growing tradition within the science of interrogation. The tradition emphasizes creating a working relationship between the interviewer and the informant, while previous interrogation work has valued a more formal approach. The Mockingbird-tactic is developed to elicit specific information from the informant without the informant perceiving that this information has been shared. The aim of this study is to compare the Mockingbird-tactic against the tactic of asking an explicit question with regards to which information that is elicited. The aim of the study is also to compare the two tactics regarding the informant's perception of what information has been requested and shared. The research questions was formulated as follows: "Which of the interrogation tactics is more effective when it comes to generating specific information?" and "How does the tactics affect the informants perception of requested and shared information?" A total of 53 individuals participated in the experiment that was followed up by a verbal questionnaire. The data was analyzed with Chi2-tests and showed that asking an explicit question was more effective regarding generating specific information. The effectiveness of the Mockingbird tactic was dependent on the topic which the confederate was looking for. The informant’s perception of requested and shared information was not statistically analyzed due to inadequate data. Only eight out of 52 participants perceived that specific information was requested / Studiens syfte är att utforska en nyutvecklad förhörstaktik vid namn Mockingbird, vilken är sprungen ur en framväxande tradition inom forskning kring förhörstekniker. Traditionen grundar sig på att skapa en fungerande relation mellan parterna i ett förhör, medan tidigare förhörsarbete mer utgjorts av en formell intervjusituation. Mockingbird syftar till att få ut specifik information från informanten utan att informanten upplever sig ha lämnat denna specifika information. Syftet med studien är att jämföra Mockingbird-taktiken med taktiken explicit fråga med avseende på vilken information som utlämnas. Studiens syfte är även att jämföra informanternas upplevelse av efterfrågad och lämnad information vid de två taktikerna. Frågorna "Vilken av förhörstaktikerna är mest effektiv för att generera specifik information?" och "Hur påverkar de två olika taktikerna informanternas upplevelse av efterfrågad och lämnad information?" undersöktes med hjälp av ett experiment och uppföljande samtal i form av muntliga enkäter, med 53 deltagare. Materialet analyserades med hjälp av Chi-2-test vilka visade att den explicita frågan var mest effektiv i att generera specifik information. Vid användande av Mockingbird-taktiken var ämnet avgörande för svarsbenägenheten. Informanternas upplevelse av efterfrågad och lämnad information analyserades inte statistiskt på grund av otillräcklig data. Endast åtta av 52 försökspersoner upplevde att specifik information efterfrågades.