Back to search

McMaster University

A Transformer-Based Multi-Agent Reinforcement Learning Framework for Asynchronous Mission Planning of Heterogeneous Robotic Teams

Abstract

dc:description.abstract

Coordinating heterogeneous teams of mobile robots in complex, partially observable environments remains a significant challenge, particularly when operating under communication constraints and asynchronous decision-making requirements. This dissertation addresses these challenges by introducing a unified framework for scalable, robust, and distributed mission planning. First, we formulate the problem as a Class-based Macro-Action Decentralized Partially Observable Markov Decision Process (CMacDec-POMDP). This novel theoretical model formally accommodates agent heterogeneity and asynchronous execution, extending the Multi-Agent Advantage Decomposition Theorem to temporally extended actions to guarantee monotonic policy improvement. Building on this foundation, we propose the Cooperative and Asynchronous Transformer-based Mission Planning (CATMiP) framework for navigation of heterogeneous mobile robots using Multi-Agent Reinforcement Learning (MARL). CATMiP integrates a sequential decision-making scheme with graph attention mechanisms to enable robust coordination within dynamic ad-hoc communication neighborhoods. We implement two distinct policy architectures within this framework: the Asynchronous Multi-Agent Transformer (AMAT) for scenarios where collaborative mapping is feasible, and the Memory-augmented Asynchronous Class-based Multi-Agent Transformer (MACMAT) for mapless navigation. MACMAT enables agents to navigate toward targets without forming explicit global maps by relying on a Recurrent Neural Network (RNN) to construct an implicit memory of their mission history. We investigate the efficacy of continuous-time, discrete-time, and mixed memory RNN architectures in handling the variable temporal gaps inherent in asynchronous multi-agent decision-making. Extensive evaluations in the custom MiniSAR simulator demonstrate MACMAT's reliability and efficiency in complex Search and Rescue (SAR) missions. Furthermore, the framework exhibits robust zero-shot generalization, successfully maintaining coordination despite novel environment scales, unseen team compositions, dynamic obstacles, and severe communication or localization failures without retraining.

Degree

thesis:*
Department dc:contributor.department
Electrical and Computer Engineering
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Farjadnasab, Milad
Advisor dc:contributor.advisor
  • Sirouspour, Shahin

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
McMaster University
Base URL
macsphere.mcmaster.ca/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Farjadnasab, Milad. A Transformer-Based Multi-Agent Reinforcement Learning Framework for Asynchronous Mission Planning of Heterogeneous Robotic Teams. 2026. https://hdl.handle.net/11375/33307