No description available.

No description available.

For zero-sum games, we have efficient solution techniques. Unfortunately, there are interesting games that are too large to solve. Here, a popular approach is to solve an abstract game that models the original game. We assume that more accurate the abstract games result in stronger strategies. There is substantial evidence to support this assumption. We begin by formalizing abstraction and refinement, a notion of expressive power for abstractions. We then show the assumption fails to hold under two criteria. The first is exploitability, which measures performance in the worst-case. The second is called the domination value, which measures how many mistakes a strategy makes. Despite these pathologies, we notice that larger strategies tend to make fewer mistakes and perform better in tournaments. Finally, we introduce strategy grafting, a technique that uses sub-game decomposition, which allow us to create good strategies in much larger spaces than previously possible.

We study maximum entropy correlated equilibria in (multi-player)games and provide two gradient-based algorithms that are guaranteedto converge to such equilibria. Although we do not provideconvergence rates for these algorithms, they do have strong connectionsto other algorithms (such as iterative scaling) which are effectiveheuristics for tasks such as statistical estimation.

This thesis formalizes a model of bounded rationality in extensive-form games called game-playing schemata. In this model, the strategies are repre- sented by a structure consisting of a deterministic finite automaton and two computational functions. The automaton represents a structured memory of the player, while the functions represent the ability of the player to identify efficient abstractions of the game. Together, the schema is a realization of a pure strategy which can be implemented by a player in order to play a given game. The thesis shows how to construct correctly playing schema for every pure strategy in any multi-player extensive-form game with perfect recall and how to evaluate its complexity. It proves that equilibria in schemata strategies always exist and computing them is PPAD-hard. Moreover, for a class of efficiently representable strategies, computing MAXPAY-EFCE can be done in polynomial time. 1

No description available.

Efficient algorithms exist for finding optimal strategies in extensive-form games. However human scale problems, such as poker, are typically so large that computation of these strategies remain infeasible with current technology. State space abstraction techniques allow us to derive a smaller abstract game, in which an optimal strategy can be computed and then used in the real game. This thesis introduces state of the art abstraction techniques. Most of these techniques do not deal with public information. We present a new automatic public state space abstraction technique. We examine the quality of this technique in the domain of poker. Our experimental results show that the new technique brings significant performance improvement. Powered by TCPDF (www.tcpdf.org)

This thesis investigates decision-making in two-player imperfect information games against opponents whose actions can affect our rewards, and whose strategies may be based on memories of interaction, or may be changing, or both. The focus is on modelling these dynamic opponents, and using the models to learn high-reward strategies. The main contributions of this work are: 1. An approach to learn high-reward strategies in small simultaneous-move games against these opponents. This is done by using a model of the opponent learnt from sequence prediction, with (possibly discounted) rewards learnt from reinforcement learning, to lookahead using explicit tree search. Empirical results show that this gains higher average rewards per game than state-of-the-art reinforcement learning agents in three simultaneous-move games. They also show that several sequence prediction methods model these opponents effectively, supporting the idea of using them from areas such as data compression and string matching; 2. An online expectation-maximisation algorithm that infers an agent's hidden information based on its behaviour in imperfect information games; 3. An approach to learn high-reward strategies in medium-size sequential-move poker games against these opponents. This is done by using a model of the opponent learnt from sequence prediction, which needs its hidden information (inferred by the online expectation-maximisation algorithm), to train a state-of-the-art no-regret learning algorithm by simulating games between the algorithm and the model. Empirical results show that this improves the no-regret learning algorithm's rewards when playing against popular and state-of-the-art algorithms in two simplified poker games; 4. Demonstrating that several change detection methods can effectively model changing categorical distributions with experimental results comparing their accuracies to empirical distributions. These results also show that their models can be used to outperform state-of-the-art reinforcement learning agents in two simultaneous-move games. This supports the idea of modelling changing opponent strategies with change detection methods; 5. Experimental results for the self-play convergence to mixed strategy Nash equilibria of the empirical distributions of plays of sequence prediction and change detection methods. The results show that they converge faster, and in more cases for change detection, than fictitious play.