University of Southampton
Scalable and efficient incentive-based traffic management using multi-agent reinforcement learning
Abstract
dc:description.abstractModern urban transportation networks are evolving into complex, decentralised systems of interacting autonomous agents. This paradigm shift introduces pervasive non-stationarity, where the adaptive behaviour of individual agents continually alters the environment for all others, fundamentally challenging classical control paradigms and creating a critical need for robust and scalable multi-agent solutions.<br/><br/>In this thesis, we address the problem of non-stationarity across the full spectrum of multi-agent interaction, from implicit cooperation to explicit competition, identifying that the key to managing these systems lies in the principled design of local incentives to steer decentralised behaviour towards globally efficient and equitable outcomes. To validate this, we investigate two distinct but complementary domains within urban traffic management. Firstly, within the context of decentralised Traffic Signal Control, we explore how local reward function design can serve as a mechanism for inducing implicit cooperation among independent intersection agents, testing the ability of fairness-oriented and hybrid rewards to mitigate non-stationarity in realistic, non-grid networks. <br/><br/>Secondly, to address the challenge of explicit competition, we introduce and formalise the Multi-Market Routing Problem, a novel game-theoretic framework. The Multi-Market Routing Problem models a market of self-interested, profit-driven route providers who learn to set adaptive prices for a population of transportation users with heterogeneous preferences. We formally establish the theoretical properties of this problem, including the existence of pure-strategy player-side equilibria, the welfare gap between strategic play and the social optimum, and the NP-hardness of computing the social optimum in general networks, and then develop a specifically modified variant of Proximal Policy Optimisation to enable agents to learn stable and effective pricing policies under the uncertainty and non-stationarity of the online setting.<br/><br/>The findings demonstrate the power of incentive engineering in multi-agent systems. Within Traffic Signal Control, we demonstrate that fairness-based and hybrid reward functions are superior mechanisms for mitigating the problem of non-stationarity, significantly outperforming traditional efficiency-based metrics. Within the Multi-Market Routing Problem, our adaptive pricing agents successfully discover near-optimal pricing policies that not only approach the performance bounds of an offline oracle but also converge to theoretically sound game-theoretic equilibria. These policies effectively manage congestion, reduce travel times, and improve the equity of outcomes for users, all while maintaining transportation provider profitability. This thesis establishes that decentralised multi-agent Reinforcement Learning guided by carefully designed local incentives is a viable and powerful paradigm for the scalable and fair management of real-world transportation systems and demonstrates that robust, system-wide coordination and efficiency can emerge from the guided interactions of independent agents, offering an effective alternative to centralised control.
Degree
thesis:*- Name dc:type.qualificationname
- Ph.D.
- Level dc:type.qualificationlevel
- doctoral
- Grantor dc:publisher.institution
- University of Southampton
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Koohy, Behrad
- Advisors dc:contributor.advisor
-
- Stein, Sebastian
- Gerding, Enrico
- Yazdanpanah, Vahid
- ORCID person:identifier.orcid
Rights
- Language dc:language
- English, eng
Identifiers
dc:identifier.*- DOI dc:identifier.doi
- https://doi.org/10.5258/SOTON/PG/T237