Policy Hill Climbing (PHC) is a reinforcement learning algorithm that extends Q-learning to learn probabilistic policies for multi-agent games. WoLF-PHC extends PHC with the "win or learn fast" principle. A proof that PHC will diverge in self-play when playing Shapley's game is given, and WoLF-PHC is shown empirically to diverge as well. Various WoLF-PHC based modifications were created, evaluated, and compared in an attempt to obtain convergence to the single shot Nash equilibrium when playing Shapley's game in self-play without using more information than WoLF-PHC uses. Partial Commitment WoLF-PHC (PCWoLF-PHC), which performs best on Shapley's game, is tested on other matrix games and shown to produce satisfactory results.

We propose a new classification for multi-agent learning algorithms, with each league of players characterized by both their possible strategies and possible beliefs. Using this classification, we review the optimality of existing algorithms and discuss some insights that can be gained. We propose an incremental improvement to the existing algorithms that seems to achieve average payoffs that are at least the Nash equilibrium payoffs in the long-run against fair opponents. / Singapore-MIT Alliance (SMA)

With a significant number of states in the U.S. and countries around the world trading electricity in restructured markets, a sizeable proportion of capacity expansion in the future will have to take place in market-based environments. However, since a majority of the literature on capacity expansion is focused on regulated market structures, there is a critical need for comprehensive capacity expansion models targeting restructured markets. In this research, we develop a two-level game-theoretic model, and a novel solution algorithm that incorporates risk due to volatilities in profit (via CVaR), to obtain multi-period, multi-player capacity expansion plans. To solve the matrix games that arise in the generation expansion planning (GEP) model, we first develop a novel value function approximation based reinforcement learning (RL) algorithm. Currently there exist only mathematical programming based solution approaches for two player games and the N-player extensions in literature still have several unresolved computational issues. Therefore, there is a critical void in literature for finding solutions of N-player matrix games. The RL-based approach we develop in this research presents itself as a viable computational alternative. The solution approach for matrix games will also serve a much broader purpose of being able to solve a larger class of problems known as stochastic games. This RL-based algorithm is used in our two-tier game-theoretic approach for obtaining generation expansion strategies. Our unique contributions to the GEP literature include the explicit consideration of risk due to volatilities in profit and individual risk preference of generators. We also consider transmission constraints, multi-year planning horizon, and multiple generation technologies. The applicability of the twotier model is demonstrated using a sample power network from PowerWorld software. A detailed analysis of the model is performed, which examines the results with respect to the nature of Nash equilibrium solutions obtained, nodal prices, factors affecting nodal prices, potential for market power, and variations in risk preferences of investors. Future research directions include the incorporation of comprehensive cap-and-trade and renewable portfolio standards components in the GEP model.

The thesis contains a textbook for gifted secondary school students. Its aim is to give to these students (or to their teachers) a Czech-written text covering fundamental principles in the field of game theory. In the first part we introduce the combinatorial games and some elementary methods of their solution. The second part is devoted to the game of Nim, to the Sprague-Grundy function and to the sums of the combinatorial games. It also contains a necessary introduction to the binary numeral system. In the third part we present the concept of matrix and bimatrix games. There is a lot of exercises and examples in the textbook. At the end we bring solutions to the most of them, providing the active reader with the opportunity of checking their own solutions.