Back to search

University of British Columbia

Empirically evaluating multiagent reinforcement learning algorithms in repeated games

Abstract

dc:description

This dissertation presents a platform for running experiments on multiagent reinforcement learning algorithms and an empirical evaluation that was conducted on the platform. The setting under consideration is game theoretic in which a single normal form game is repeatedly played. There has been a large body of work focusing on introducing new algorithms to achieve certain goals such as guaranteeing values in a game, converging to a Nash equilibrium or minimizing total regret. Currently, we have an understanding of how some of these algorithms work in limited settings, but lack a broader understanding of which algorithms perform well against each other and how they perform on a larger variety of games. We describe our development of a platform that allows large scale tests to be run, where multiple algorithms are played against one another on a variety of games. The platform has a set of builtin metrics that can be used to measure the performance of an algorithm, including convergence to a Nash equilibrium, regret, reward and number of wins. Visualising the results of the test can be automatically achieved through the platform, with all interaction taking place through graphical user interfaces. We also present the results of an empirical test that to our knowledge includes the largest combination of game instances and algorithms used in the multiagent learning literature. To demonstrate the usefulness of the platform, we provide evidence for a number of claims and hypotheses. This includes claims related to convergence to a Nash equilibrium, reward, regret and best response metrics and claims dealing with estimating an opponent's strategy. Some of our claims include that (1) no algorithm does best across all metrics and over all opponents, (2) algorithms do not often converge to an exact Nash equilibrium, but (3) do often reach a small window around a Nash equilibrium, (4) there is no apparent link between converging to a Nash equilibrium and obtaining high reward and (5) there is no linear trend between reward and the size of the game for any agent. The two major contributions of this work are a software platform for running large experimental tests and empirical results that provide insight into the performance of various algorithms.

Degree

thesis:*
Name thesis:degree_name
Master of Science - MSc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2005

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lipson, Asher G.

Rights

dc:rights
Statement dc:rights
  • For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/16633

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Lipson, Asher G.. Empirically evaluating multiagent reinforcement learning algorithms in repeated games. master's thesis, University of British Columbia, 2005. http://hdl.handle.net/2429/16633