McMaster University
A Transformer-Based Multi-Agent Reinforcement Learning Framework for Asynchronous Mission Planning of Heterogeneous Robotic Teams
Abstract
dc:description.abstractCoordinating heterogeneous teams of mobile robots in complex, partially observable environments remains a significant challenge, particularly when operating under communication constraints and asynchronous decision-making requirements. This dissertation addresses these challenges by introducing a unified framework for scalable, robust, and distributed mission planning. First, we formulate the problem as a Class-based Macro-Action Decentralized Partially Observable Markov Decision Process (CMacDec-POMDP). This novel theoretical model formally accommodates agent heterogeneity and asynchronous execution, extending the Multi-Agent Advantage Decomposition Theorem to temporally extended actions to guarantee monotonic policy improvement. Building on this foundation, we propose the Cooperative and Asynchronous Transformer-based Mission Planning (CATMiP) framework for navigation of heterogeneous mobile robots using Multi-Agent Reinforcement Learning (MARL). CATMiP integrates a sequential decision-making scheme with graph attention mechanisms to enable robust coordination within dynamic ad-hoc communication neighborhoods. We implement two distinct policy architectures within this framework: the Asynchronous Multi-Agent Transformer (AMAT) for scenarios where collaborative mapping is feasible, and the Memory-augmented Asynchronous Class-based Multi-Agent Transformer (MACMAT) for mapless navigation. MACMAT enables agents to navigate toward targets without forming explicit global maps by relying on a Recurrent Neural Network (RNN) to construct an implicit memory of their mission history. We investigate the efficacy of continuous-time, discrete-time, and mixed memory RNN architectures in handling the variable temporal gaps inherent in asynchronous multi-agent decision-making. Extensive evaluations in the custom MiniSAR simulator demonstrate MACMAT's reliability and efficiency in complex Search and Rescue (SAR) missions. Furthermore, the framework exhibits robust zero-shot generalization, successfully maintaining coordination despite novel environment scales, unseen team compositions, dynamic obstacles, and severe communication or localization failures without retraining.
Degree
thesis:*- Department dc:contributor.department
- Electrical and Computer Engineering
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Farjadnasab, Milad
- Advisor dc:contributor.advisor
-
- Sirouspour, Shahin
Rights
dc:rights- Statement dc:rights
-
- Attribution 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Identifier URI
- https://doi.org/10.71548/169