Back to search

University of Southampton

Scalable and efficient incentive-based traffic management using multi-agent reinforcement learning

Abstract

dc:description.abstract

Modern urban transportation networks are evolving into complex, decentralised systems of interacting autonomous agents. This paradigm shift introduces pervasive non-stationarity, where the adaptive behaviour of individual agents continually alters the environment for all others, fundamentally challenging classical control paradigms and creating a critical need for robust and scalable multi-agent solutions.<br/><br/>In this thesis, we address the problem of non-stationarity across the full spectrum of multi-agent interaction, from implicit cooperation to explicit competition, identifying that the key to managing these systems lies in the principled design of local incentives to steer decentralised behaviour towards globally efficient and equitable outcomes. To validate this, we investigate two distinct but complementary domains within urban traffic management. Firstly, within the context of decentralised Traffic Signal Control, we explore how local reward function design can serve as a mechanism for inducing implicit cooperation among independent intersection agents, testing the ability of fairness-oriented and hybrid rewards to mitigate non-stationarity in realistic, non-grid networks. <br/><br/>Secondly, to address the challenge of explicit competition, we introduce and formalise the Multi-Market Routing Problem, a novel game-theoretic framework. The Multi-Market Routing Problem models a market of self-interested, profit-driven route providers who learn to set adaptive prices for a population of transportation users with heterogeneous preferences. We formally establish the theoretical properties of this problem, including the existence of pure-strategy player-side equilibria, the welfare gap between strategic play and the social optimum, and the NP-hardness of computing the social optimum in general networks, and then develop a specifically modified variant of Proximal Policy Optimisation to enable agents to learn stable and effective pricing policies under the uncertainty and non-stationarity of the online setting.<br/><br/>The findings demonstrate the power of incentive engineering in multi-agent systems. Within Traffic Signal Control, we demonstrate that fairness-based and hybrid reward functions are superior mechanisms for mitigating the problem of non-stationarity, significantly outperforming traditional efficiency-based metrics. Within the Multi-Market Routing Problem, our adaptive pricing agents successfully discover near-optimal pricing policies that not only approach the performance bounds of an offline oracle but also converge to theoretically sound game-theoretic equilibria. These policies effectively manage congestion, reduce travel times, and improve the equity of outcomes for users, all while maintaining transportation provider profitability. This thesis establishes that decentralised multi-agent Reinforcement Learning guided by carefully designed local incentives is a viable and powerful paradigm for the scalable and fair management of real-world transportation systems and demonstrates that robust, system-wide coordination and efficiency can emerge from the guided interactions of independent agents, offering an effective alternative to centralised control.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D.
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Southampton
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Koohy, Behrad
Advisors dc:contributor.advisor
  • Stein, Sebastian
  • Gerding, Enrico
  • Yazdanpanah, Vahid

Rights

Language dc:language
English, eng

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
University of Southampton
Base URL
eprints.soton.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Koohy, Behrad. Scalable and efficient incentive-based traffic management using multi-agent reinforcement learning. doctoral thesis, University of Southampton, 2026. https://doi.org/10.5258/SOTON/PG/T237