{"id":{"repo_id":"ets-quebec","oai_identifier":"oai:espace.etsmtl.ca:4024"},"canonical_url":"https://search.ndltd.org/etd/ets-quebec/oai:espace.etsmtl.ca:4024","repository":{"repo_id":"ets-quebec","name":"ETS (Quebec)","base_url":"https://espace.etsmtl.ca/cgi/oai2"},"display":{"title":"Reinforcement learning-driven adaptive spectrum utilization in cognitive B5G networks","abstract":"In recent years, the growth of wireless services and connected devices increased the demand for radio spectrum. Static spectrum allocation could not fully address this demand, because licensed bands may remain underused across time and location. Cognitive radio networks (CRNs) allowed secondary users (SUs) to access available spectrum while preserving primary user (PU) priority. In these networks, efficient spectrum use required reliable sensing, effective access decisions, infrastructure-side coordination, resilience, and trust management. Accordingly, in the research reported in the present thesis, we improved spectrum utilization by addressing cooperative spectrum sensing, SU-side spectrum access, and coordination between spectrum-consuming and spectrum-granting nodes. These tasks involved sequential decisions under uncertainty and coupled interactions across users and network tiers. Therefore, we used reinforcement learning to support adaptive sensing, access, and coordination decisions. First, we developed Partially Cooperative Multi-Agent Reinforcement Learning (PCMARL) for cooperative spectrum sensing in cognitive radio Internet-of-Things networks under spectrum sensing data falsification attacks. PCMARL allowed SUs to selectively cooperate and improved sensing reliability and energy efficiency without full network cooperation. Second, we extended SU-side optimization from sensing to joint sensing and access. The proposed IMPACT framework optimized sensing participation, cooperation, channel selection, and transmission power under imperfect channel state information. Since access control required both discrete decisions and continuous power control, we introduced a hybrid-action learning formulation for cooperative spectrum access. Third, we added the spectrum granting and coordination side of the network. The proposed Federated Hierarchical Reinforcement Learning (F-HRL) framework coordinated SUs, next generation Node Bs, uncrewed aerial vehicles, and high-altitude platform stations. This architecture supported grant allocation, PU priority, trust-aware scheduling, privacy-preserving federation, and service continuity during infrastructure disruption. The results revealed that improved spectrum utilization required SU-side sensing and access optimization together with granting side coordination, infrastructure resilience, federated aggregation, and trust-aware control. Overall, in this research, we developed an integrated learning-based framework for resilient spectrum management in future wireless networks.","abstract_html":"In recent years, the growth of wireless services and connected devices increased the demand for radio spectrum. Static spectrum allocation could not fully address this demand, because licensed bands may remain underused across time and location. Cognitive radio networks (CRNs) allowed secondary users (SUs) to access available spectrum while preserving primary user (PU) priority. In these networks, efficient spectrum use required reliable sensing, effective access decisions, infrastructure-side coordination, resilience, and trust management. Accordingly, in the research reported in the present thesis, we improved spectrum utilization by addressing cooperative spectrum sensing, SU-side spectrum access, and coordination between spectrum-consuming and spectrum-granting nodes. These tasks involved sequential decisions under uncertainty and coupled interactions across users and network tiers. Therefore, we used reinforcement learning to support adaptive sensing, access, and coordination decisions. First, we developed Partially Cooperative Multi-Agent Reinforcement Learning (PCMARL) for cooperative spectrum sensing in cognitive radio Internet-of-Things networks under spectrum sensing data falsification attacks. PCMARL allowed SUs to selectively cooperate and improved sensing reliability and energy efficiency without full network cooperation. Second, we extended SU-side optimization from sensing to joint sensing and access. The proposed IMPACT framework optimized sensing participation, cooperation, channel selection, and transmission power under imperfect channel state information. Since access control required both discrete decisions and continuous power control, we introduced a hybrid-action learning formulation for cooperative spectrum access. Third, we added the spectrum granting and coordination side of the network. The proposed Federated Hierarchical Reinforcement Learning (F-HRL) framework coordinated SUs, next generation Node Bs, uncrewed aerial vehicles, and high-altitude platform stations. This architecture supported grant allocation, PU priority, trust-aware scheduling, privacy-preserving federation, and service continuity during infrastructure disruption. The results revealed that improved spectrum utilization required SU-side sensing and access optimization together with granting side coordination, infrastructure resilience, federated aggregation, and trust-aware control. Overall, in this research, we developed an integrated learning-based framework for resilient spectrum management in future wireless networks.","abstract_has_math":false,"creators":["Khaf, Sadia"],"institution":"École de technologie supérieure","degree_name":"Thèse de doctorat électronique","degree_level":"phd","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2026,"date_issued":"2026-07-04","date_published":"2026-07-04","updated_at":"2026-11-18T22:11:16Z","subjects":["radio cognitive, apprentissage par renforcement, gestion du spectre, apprentissage fédéré, résilience"],"languages":["en"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["INSERT-YOUR-LIBRARY-ID-HERE4024"],"render_values":[{"text":"INSERT-YOUR-LIBRARY-ID-HERE4024","href":null,"code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":null,"outbound_label":null,"outbound_source":null,"record_url":null,"record_label":null,"record_kind":null,"record_source":null,"download_url":"https://espace.etsmtl.ca/id/eprint/4024/1/KHAF_Sadia.pdf","download_label":"Download PDF","download_source":"dc:identifier","access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://espace.etsmtl.ca/cgi/oai2?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Aespace.etsmtl.ca%3A4024","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:creator","label":"Author","values":["Khaf, Sadia"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2026-07-04"]},{"key":"dc:publisher","label":"Institution","values":["École de technologie supérieure"]},{"key":"dc:type","label":"Dc Type","values":["Electronic Thesis or Dissertation"]},{"key":"thesis:degree_level","label":"Degree Level","values":["phd"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Thèse de doctorat électronique"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["École de technologie supérieure"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["radio cognitive, apprentissage par renforcement, gestion du spectre, apprentissage fédéré, résilience"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["INSERT-YOUR-LIBRARY-ID-HERE4024","https://espace.etsmtl.ca/id/eprint/4024/1/KHAF_Sadia.pdf"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["In recent years, the growth of wireless services and connected devices increased the demand for radio spectrum. Static spectrum allocation could not fully address this demand, because licensed bands may remain underused across time and location. Cognitive radio networks (CRNs) allowed secondary users (SUs) to access available spectrum while preserving primary user (PU) priority. In these networks, efficient spectrum use required reliable sensing, effective access decisions, infrastructure-side coordination, resilience, and trust management. Accordingly, in the research reported in the present thesis, we improved spectrum utilization by addressing cooperative spectrum sensing, SU-side spectrum access, and coordination between spectrum-consuming and spectrum-granting nodes. These tasks involved sequential decisions under uncertainty and coupled interactions across users and network tiers. Therefore, we used reinforcement learning to support adaptive sensing, access, and coordination decisions. First, we developed Partially Cooperative Multi-Agent Reinforcement Learning (PCMARL) for cooperative spectrum sensing in cognitive radio Internet-of-Things networks under spectrum sensing data falsification attacks. PCMARL allowed SUs to selectively cooperate and improved sensing reliability and energy efficiency without full network cooperation. Second, we extended SU-side optimization from sensing to joint sensing and access. The proposed IMPACT framework optimized sensing participation, cooperation, channel selection, and transmission power under imperfect channel state information. Since access control required both discrete decisions and continuous power control, we introduced a hybrid-action learning formulation for cooperative spectrum access. Third, we added the spectrum granting and coordination side of the network. The proposed Federated Hierarchical Reinforcement Learning (F-HRL) framework coordinated SUs, next generation Node Bs, uncrewed aerial vehicles, and high-altitude platform stations. This architecture supported grant allocation, PU priority, trust-aware scheduling, privacy-preserving federation, and service continuity during infrastructure disruption. The results revealed that improved spectrum utilization required SU-side sensing and access optimization together with granting side coordination, infrastructure resilience, federated aggregation, and trust-aware control. Overall, in this research, we developed an integrated learning-based framework for resilient spectrum management in future wireless networks."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Reinforcement learning-driven adaptive spectrum utilization in cognitive B5G networks"]}]}],"canonical_facts":{"dc:creator":["Khaf, Sadia"],"dc:date":["2026-07-04"],"dc:description":["In recent years, the growth of wireless services and connected devices increased the demand for radio spectrum. Static spectrum allocation could not fully address this demand, because licensed bands may remain underused across time and location. Cognitive radio networks (CRNs) allowed secondary users (SUs) to access available spectrum while preserving primary user (PU) priority. In these networks, efficient spectrum use required reliable sensing, effective access decisions, infrastructure-side coordination, resilience, and trust management. Accordingly, in the research reported in the present thesis, we improved spectrum utilization by addressing cooperative spectrum sensing, SU-side spectrum access, and coordination between spectrum-consuming and spectrum-granting nodes. These tasks involved sequential decisions under uncertainty and coupled interactions across users and network tiers. Therefore, we used reinforcement learning to support adaptive sensing, access, and coordination decisions. First, we developed Partially Cooperative Multi-Agent Reinforcement Learning (PCMARL) for cooperative spectrum sensing in cognitive radio Internet-of-Things networks under spectrum sensing data falsification attacks. PCMARL allowed SUs to selectively cooperate and improved sensing reliability and energy efficiency without full network cooperation. Second, we extended SU-side optimization from sensing to joint sensing and access. The proposed IMPACT framework optimized sensing participation, cooperation, channel selection, and transmission power under imperfect channel state information. Since access control required both discrete decisions and continuous power control, we introduced a hybrid-action learning formulation for cooperative spectrum access. Third, we added the spectrum granting and coordination side of the network. The proposed Federated Hierarchical Reinforcement Learning (F-HRL) framework coordinated SUs, next generation Node Bs, uncrewed aerial vehicles, and high-altitude platform stations. This architecture supported grant allocation, PU priority, trust-aware scheduling, privacy-preserving federation, and service continuity during infrastructure disruption. The results revealed that improved spectrum utilization required SU-side sensing and access optimization together with granting side coordination, infrastructure resilience, federated aggregation, and trust-aware control. Overall, in this research, we developed an integrated learning-based framework for resilient spectrum management in future wireless networks."],"dc:format":["application/pdf"],"dc:identifier":["INSERT-YOUR-LIBRARY-ID-HERE4024","https://espace.etsmtl.ca/id/eprint/4024/1/KHAF_Sadia.pdf"],"dc:language":["en"],"dc:publisher":["École de technologie supérieure"],"dc:subject":["radio cognitive, apprentissage par renforcement, gestion du spectre, apprentissage fédéré, résilience"],"dc:title":["Reinforcement learning-driven adaptive spectrum utilization in cognitive B5G networks"],"dc:type":["Electronic Thesis or Dissertation"],"thesis:degree_level":["phd"],"thesis:degree_name":["Thèse de doctorat électronique"],"thesis:institution_name":["École de technologie supérieure"]},"updated_at":"2026-11-18T22:11:16Z"}