University of Southampton
Adaptive market making using reinforcement learning in a multi-agent market simulation
Abstract
dc:description.abstractFinancial markets are amongst the most complex and well-studied multi-agent systems. Over the past several decades, researchers have devoted considerable effort to understanding their structural and behavioural properties. However, building a simulated market environment that can replicate realistic interactive behaviour remains an ongoing challenge. This limitation restricts the scope and validity of experiments aimed at testing new trading strategies or hypotheses, particularly when simplifying assumptions, such as negligible market impact or the absence of market feedback, fail to hold.<br/><br/>This thesis addresses this challenge by building on the Agent-Based Interactive Discrete Event Simulation (ABIDES) simulation platform to create an empirically grounded and reproducible environment for studying cryptocurrency markets. First, we develop a methodology for tuning simulation parameters to replicate the stylised facts observed in Binance following the 2017 cryptocurrency boom. Second, we introduce Price-Reverting Impact Model of a cryptocurrency Exchange (PRIME), a novel configuration of the ABIDES simulator designed to produce realistic market responses to agent actions whilst retaining the ability to track an external price series. Third, we use the PRIME framework to conduct the first controlled comparison of Reinforcement Learning (RL) based market makers across various architectures in a realistic, interactive environment, establishing a benchmark for future research.<br/><br/>Our findings demonstrate that a carefully calibrated agent population comprising Zero Intelligence, Momentum, and Mean-Reversion agents can reproduce key statistical properties observed in the Bitcoin market. Furthermore, the PRIME framework successfully captures market impact behaviours described in existing and novel empirical observations, allowing for controlled but realistic experimentation of market agents in this environment. Finally, our reinforcement learning market-makers deployed on this platform reveal important insights into the trade-offs involved in state-action space design and the learning dynamics of various RL architectures. This setup also allows for a meaningful comparative evaluation of market-makers with previous work. Collectively, these contributions provide a reproducible foundation for advancing market simulation frameworks and for developing and evaluating novel reinforcement learning approaches to the market-making problem.
Degree
thesis:*- Name dc:type.qualificationname
- Ph.D.
- Level dc:type.qualificationlevel
- doctoral
- Grantor dc:publisher.institution
- University of Southampton
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Cho, Christopher Jaehoon
- Advisors dc:contributor.advisor
-
- Norman, Tim
- Nunes, Manuel
- ORCID person:identifier.orcid
Rights
- Language dc:language
- English, eng
Identifiers
dc:identifier.*- DOI dc:identifier.doi
- https://doi.org/10.5258/SOTON/PG/T316