{"id":{"repo_id":"mcmaster","oai_identifier":"oai:macsphere.mcmaster.ca:11375/33307"},"canonical_url":"https://search.ndltd.org/etd/mcmaster/oai:macsphere.mcmaster.ca:11375/33307","repository":{"repo_id":"mcmaster","name":"McMaster University","base_url":"https://macsphere.mcmaster.ca/server/oai/request"},"display":{"title":"A Transformer-Based Multi-Agent Reinforcement Learning Framework for Asynchronous Mission Planning of Heterogeneous Robotic Teams","abstract":"Coordinating heterogeneous teams of mobile robots in complex, partially observable environments remains a significant challenge, particularly when operating under communication constraints and asynchronous decision-making requirements. This dissertation addresses these challenges by introducing a unified framework for scalable, robust, and distributed mission planning. First, we formulate the problem as a Class-based Macro-Action Decentralized Partially Observable Markov Decision Process (CMacDec-POMDP). This novel theoretical model formally accommodates agent heterogeneity and asynchronous execution, extending the Multi-Agent Advantage Decomposition Theorem to temporally extended actions to guarantee monotonic policy improvement. Building on this foundation, we propose the Cooperative and Asynchronous Transformer-based Mission Planning (CATMiP) framework for navigation of heterogeneous mobile robots using Multi-Agent Reinforcement Learning (MARL). CATMiP integrates a sequential decision-making scheme with graph attention mechanisms to enable robust coordination within dynamic ad-hoc communication neighborhoods. We implement two distinct policy architectures within this framework: the Asynchronous Multi-Agent Transformer (AMAT) for scenarios where collaborative mapping is feasible, and the Memory-augmented Asynchronous Class-based Multi-Agent Transformer (MACMAT) for mapless navigation. MACMAT enables agents to navigate toward targets without forming explicit global maps by relying on a Recurrent Neural Network (RNN) to construct an implicit memory of their mission history. We investigate the efficacy of continuous-time, discrete-time, and mixed memory RNN architectures in handling the variable temporal gaps inherent in asynchronous multi-agent decision-making. Extensive evaluations in the custom MiniSAR simulator demonstrate MACMAT's reliability and efficiency in complex Search and Rescue (SAR) missions. Furthermore, the framework exhibits robust zero-shot generalization, successfully maintaining coordination despite novel environment scales, unseen team compositions, dynamic obstacles, and severe communication or localization failures without retraining.","abstract_html":"Coordinating heterogeneous teams of mobile robots in complex, partially observable environments remains a significant challenge, particularly when operating under communication constraints and asynchronous decision-making requirements. This dissertation addresses these challenges by introducing a unified framework for scalable, robust, and distributed mission planning. First, we formulate the problem as a Class-based Macro-Action Decentralized Partially Observable Markov Decision Process (CMacDec-POMDP). This novel theoretical model formally accommodates agent heterogeneity and asynchronous execution, extending the Multi-Agent Advantage Decomposition Theorem to temporally extended actions to guarantee monotonic policy improvement. Building on this foundation, we propose the Cooperative and Asynchronous Transformer-based Mission Planning (CATMiP) framework for navigation of heterogeneous mobile robots using Multi-Agent Reinforcement Learning (MARL). CATMiP integrates a sequential decision-making scheme with graph attention mechanisms to enable robust coordination within dynamic ad-hoc communication neighborhoods. We implement two distinct policy architectures within this framework: the Asynchronous Multi-Agent Transformer (AMAT) for scenarios where collaborative mapping is feasible, and the Memory-augmented Asynchronous Class-based Multi-Agent Transformer (MACMAT) for mapless navigation. MACMAT enables agents to navigate toward targets without forming explicit global maps by relying on a Recurrent Neural Network (RNN) to construct an implicit memory of their mission history. We investigate the efficacy of continuous-time, discrete-time, and mixed memory RNN architectures in handling the variable temporal gaps inherent in asynchronous multi-agent decision-making. Extensive evaluations in the custom MiniSAR simulator demonstrate MACMAT&#x27;s reliability and efficiency in complex Search and Rescue (SAR) missions. Furthermore, the framework exhibits robust zero-shot generalization, successfully maintaining coordination despite novel environment scales, unseen team compositions, dynamic obstacles, and severe communication or localization failures without retraining.","abstract_has_math":false,"creators":["Farjadnasab, Milad"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":"Electrical and Computer Engineering","school":null,"contributors":[],"advisors":["Sirouspour, Shahin"],"committee_chairs":[],"committee_members":[],"year":2026,"date_issued":"2026","date_published":"2026","updated_at":"2026-11-18T19:39:27Z","subjects":[],"languages":["en"],"rights":["Attribution 4.0 International"],"rights_urls":["http://creativecommons.org/licenses/by/4.0/"],"identifier_entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://doi.org/10.71548/169"],"render_values":[{"text":"https://doi.org/10.71548/169","href":"https://doi.org/10.71548/169","code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/11375/33307","outbound_label":"Handle","outbound_source":"dc:identifier.uri","record_url":"https://hdl.handle.net/11375/33307","record_label":"Repository record","record_kind":"handle","record_source":"dc:identifier.uri","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://macsphere.mcmaster.ca/server/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Amacsphere.mcmaster.ca%3A11375%2F33307","prefix":"dim"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Sirouspour, Shahin"]},{"key":"dc:contributor.department","label":"Department","values":["Electrical and Computer Engineering"]},{"key":"dc:creator","label":"Author","values":["Farjadnasab, Milad"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2026-06-29T15:03:53Z"]},{"key":"dc:date.issued","label":"Date","values":["2026"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Attribution 4.0 International"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://creativecommons.org/licenses/by/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/11375/33307","https://doi.org/10.71548/169"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Coordinating heterogeneous teams of mobile robots in complex, partially observable environments remains a significant challenge, particularly when operating under communication constraints and asynchronous decision-making requirements. This dissertation addresses these challenges by introducing a unified framework for scalable, robust, and distributed mission planning. First, we formulate the problem as a Class-based Macro-Action Decentralized Partially Observable Markov Decision Process (CMacDec-POMDP). This novel theoretical model formally accommodates agent heterogeneity and asynchronous execution, extending the Multi-Agent Advantage Decomposition Theorem to temporally extended actions to guarantee monotonic policy improvement. Building on this foundation, we propose the Cooperative and Asynchronous Transformer-based Mission Planning (CATMiP) framework for navigation of heterogeneous mobile robots using Multi-Agent Reinforcement Learning (MARL). CATMiP integrates a sequential decision-making scheme with graph attention mechanisms to enable robust coordination within dynamic ad-hoc communication neighborhoods. We implement two distinct policy architectures within this framework: the Asynchronous Multi-Agent Transformer (AMAT) for scenarios where collaborative mapping is feasible, and the Memory-augmented Asynchronous Class-based Multi-Agent Transformer (MACMAT) for mapless navigation. MACMAT enables agents to navigate toward targets without forming explicit global maps by relying on a Recurrent Neural Network (RNN) to construct an implicit memory of their mission history. We investigate the efficacy of continuous-time, discrete-time, and mixed memory RNN architectures in handling the variable temporal gaps inherent in asynchronous multi-agent decision-making. Extensive evaluations in the custom MiniSAR simulator demonstrate MACMAT's reliability and efficiency in complex Search and Rescue (SAR) missions. Furthermore, the framework exhibits robust zero-shot generalization, successfully maintaining coordination despite novel environment scales, unseen team compositions, dynamic obstacles, and severe communication or localization failures without retraining."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Doctor of Philosophy (PhD)"]},{"key":"dc:title","label":"Title","values":["A Transformer-Based Multi-Agent Reinforcement Learning Framework for Asynchronous Mission Planning of Heterogeneous Robotic Teams"]},{"key":"others:access_status","label":"Others Access Status","values":["open.access"]}]}],"canonical_facts":{"dc:contributor.advisor":["Sirouspour, Shahin"],"dc:contributor.department":["Electrical and Computer Engineering"],"dc:creator":["Farjadnasab, Milad"],"dc:date.accessioned":["2026-06-29T15:03:53Z"],"dc:date.issued":["2026"],"dc:description.abstract":["Coordinating heterogeneous teams of mobile robots in complex, partially observable environments remains a significant challenge, particularly when operating under communication constraints and asynchronous decision-making requirements. This dissertation addresses these challenges by introducing a unified framework for scalable, robust, and distributed mission planning. First, we formulate the problem as a Class-based Macro-Action Decentralized Partially Observable Markov Decision Process (CMacDec-POMDP). This novel theoretical model formally accommodates agent heterogeneity and asynchronous execution, extending the Multi-Agent Advantage Decomposition Theorem to temporally extended actions to guarantee monotonic policy improvement. Building on this foundation, we propose the Cooperative and Asynchronous Transformer-based Mission Planning (CATMiP) framework for navigation of heterogeneous mobile robots using Multi-Agent Reinforcement Learning (MARL). CATMiP integrates a sequential decision-making scheme with graph attention mechanisms to enable robust coordination within dynamic ad-hoc communication neighborhoods. We implement two distinct policy architectures within this framework: the Asynchronous Multi-Agent Transformer (AMAT) for scenarios where collaborative mapping is feasible, and the Memory-augmented Asynchronous Class-based Multi-Agent Transformer (MACMAT) for mapless navigation. MACMAT enables agents to navigate toward targets without forming explicit global maps by relying on a Recurrent Neural Network (RNN) to construct an implicit memory of their mission history. We investigate the efficacy of continuous-time, discrete-time, and mixed memory RNN architectures in handling the variable temporal gaps inherent in asynchronous multi-agent decision-making. Extensive evaluations in the custom MiniSAR simulator demonstrate MACMAT's reliability and efficiency in complex Search and Rescue (SAR) missions. Furthermore, the framework exhibits robust zero-shot generalization, successfully maintaining coordination despite novel environment scales, unseen team compositions, dynamic obstacles, and severe communication or localization failures without retraining."],"dc:description.degree":["Doctor of Philosophy (PhD)"],"dc:identifier.uri":["https://hdl.handle.net/11375/33307","https://doi.org/10.71548/169"],"dc:language.iso":["en"],"dc:rights":["Attribution 4.0 International"],"dc:rights.uri":["http://creativecommons.org/licenses/by/4.0/"],"dc:title":["A Transformer-Based Multi-Agent Reinforcement Learning Framework for Asynchronous Mission Planning of Heterogeneous Robotic Teams"],"dc:type":["Thesis"],"others:access_status":["open.access"]},"updated_at":"2026-11-18T19:39:27Z"}