{"id":{"repo_id":"mcmaster","oai_identifier":"oai:macsphere.mcmaster.ca:11375/32597"},"canonical_url":"https://search.ndltd.org/etd/mcmaster/oai:macsphere.mcmaster.ca:11375/32597","repository":{"repo_id":"mcmaster","name":"McMaster University","base_url":"https://macsphere.mcmaster.ca/server/oai/request"},"display":{"title":"Neuroplasticity in Genetic Programming Agents for Adaptive and Continual Decision Making","abstract":"Dynamically decomposing complex tasks into reusable sub-policies remains a core challenge in Reinforcement Learning. Tangled Program Graphs, a genetic-programming framework for general-purpose machine learning (applied here to reinforcement learning), addresses this by evolving connections between different agents in order to break down complex problems into manageable sub-problems. Inspired by memetic algorithms, which accelerate evolutionary search through agentic refinement, we introduce Neuro-Tangled Program Graphs. This biologically grounded extension utilizes hierarchical plasticity within the structure of an agent, applying a homeostatic rule at the initial decision edges and a competitive Oja-style update in each subsequent decision edge. Evaluated on both a static and dynamic variant of the MuJoCo Ant environment, this approach yields higher peak returns and evolves with 59-88% fewer mean effective instructions used per step, demonstrating stronger performance and a more compact search. Next, we add an TD-style online value baseline and eligibility traces to stabilize and distribute dense step-wise rewards over time, sharpening temporal updates within each agent. We then examine how trace length and a per-team plasticity decay factor shape learning dynamics. To set these, we compare end-to-end evolutionary tuning with MAP-Elites using a multi-archive that explores (trace length x decay). The benefits of reward modulation are then tested for with TPG and NeuroTPG variants on a customized static and dynamic maze environment. This addition show a consistently better performance across all seeds and also a more interpretable final structure. Overall, our findings highlight the vital role of a local search within population search algorithms. Our studies hope to open a new avenue to gradient-free memetic algorithms which offer many benefits and opportunities from various already developed field of studies.","abstract_html":"Dynamically decomposing complex tasks into reusable sub-policies remains a core challenge in Reinforcement Learning. Tangled Program Graphs, a genetic-programming framework for general-purpose machine learning (applied here to reinforcement learning), addresses this by evolving connections between different agents in order to break down complex problems into manageable sub-problems. Inspired by memetic algorithms, which accelerate evolutionary search through agentic refinement, we introduce Neuro-Tangled Program Graphs. This biologically grounded extension utilizes hierarchical plasticity within the structure of an agent, applying a homeostatic rule at the initial decision edges and a competitive Oja-style update in each subsequent decision edge. Evaluated on both a static and dynamic variant of the MuJoCo Ant environment, this approach yields higher peak returns and evolves with 59-88% fewer mean effective instructions used per step, demonstrating stronger performance and a more compact search. Next, we add an TD-style online value baseline and eligibility traces to stabilize and distribute dense step-wise rewards over time, sharpening temporal updates within each agent. We then examine how trace length and a per-team plasticity decay factor shape learning dynamics. To set these, we compare end-to-end evolutionary tuning with MAP-Elites using a multi-archive that explores (trace length x decay). The benefits of reward modulation are then tested for with TPG and NeuroTPG variants on a customized static and dynamic maze environment. This addition show a consistently better performance across all seeds and also a more interpretable final structure. Overall, our findings highlight the vital role of a local search within population search algorithms. Our studies hope to open a new avenue to gradient-free memetic algorithms which offer many benefits and opportunities from various already developed field of studies.","abstract_has_math":false,"creators":["Naqvi, Ali"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":"Computing and Software","school":null,"contributors":[],"advisors":["Kelly, Stephen"],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-11","date_published":"2025-11","updated_at":"2026-11-18T19:38:29Z","subjects":["Genetic Programming","Artificial Intelligence","Evolutionary Computation","Digital Evolution"],"languages":["en"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://doi.org/10.71548/21070"],"render_values":[{"text":"https://doi.org/10.71548/21070","href":"https://doi.org/10.71548/21070","code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/11375/32597","outbound_label":"Handle","outbound_source":"dc:identifier.uri","record_url":"http://hdl.handle.net/11375/32597","record_label":"Repository record","record_kind":"handle","record_source":"dc:identifier.uri","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://macsphere.mcmaster.ca/server/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Amacsphere.mcmaster.ca%3A11375%2F32597","prefix":"dim"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Kelly, Stephen"]},{"key":"dc:contributor.department","label":"Department","values":["Computing and Software"]},{"key":"dc:creator","label":"Author","values":["Naqvi, Ali"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2025-10-27T17:34:56Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2025-10-27T17:34:56Z"]},{"key":"dc:date.issued","label":"Date","values":["2025-11"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Genetic Programming","Artificial Intelligence","Evolutionary Computation","Digital Evolution"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["http://hdl.handle.net/11375/32597","https://doi.org/10.71548/21070"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Dynamically decomposing complex tasks into reusable sub-policies remains a core challenge in Reinforcement Learning. Tangled Program Graphs, a genetic-programming framework for general-purpose machine learning (applied here to reinforcement learning), addresses this by evolving connections between different agents in order to break down complex problems into manageable sub-problems. Inspired by memetic algorithms, which accelerate evolutionary search through agentic refinement, we introduce Neuro-Tangled Program Graphs. This biologically grounded extension utilizes hierarchical plasticity within the structure of an agent, applying a homeostatic rule at the initial decision edges and a competitive Oja-style update in each subsequent decision edge. Evaluated on both a static and dynamic variant of the MuJoCo Ant environment, this approach yields higher peak returns and evolves with 59-88% fewer mean effective instructions used per step, demonstrating stronger performance and a more compact search. Next, we add an TD-style online value baseline and eligibility traces to stabilize and distribute dense step-wise rewards over time, sharpening temporal updates within each agent. We then examine how trace length and a per-team plasticity decay factor shape learning dynamics. To set these, we compare end-to-end evolutionary tuning with MAP-Elites using a multi-archive that explores (trace length x decay). The benefits of reward modulation are then tested for with TPG and NeuroTPG variants on a customized static and dynamic maze environment. This addition show a consistently better performance across all seeds and also a more interpretable final structure. Overall, our findings highlight the vital role of a local search within population search algorithms. Our studies hope to open a new avenue to gradient-free memetic algorithms which offer many benefits and opportunities from various already developed field of studies."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Master of Science (MSc)"]},{"key":"dc:title","label":"Title","values":["Neuroplasticity in Genetic Programming Agents for Adaptive and Continual Decision Making"]},{"key":"others:access_status","label":"Others Access Status","values":["open.access"]}]}],"canonical_facts":{"dc:contributor.advisor":["Kelly, Stephen"],"dc:contributor.department":["Computing and Software"],"dc:creator":["Naqvi, Ali"],"dc:date.accessioned":["2025-10-27T17:34:56Z"],"dc:date.available":["2025-10-27T17:34:56Z"],"dc:date.issued":["2025-11"],"dc:description.abstract":["Dynamically decomposing complex tasks into reusable sub-policies remains a core challenge in Reinforcement Learning. Tangled Program Graphs, a genetic-programming framework for general-purpose machine learning (applied here to reinforcement learning), addresses this by evolving connections between different agents in order to break down complex problems into manageable sub-problems. Inspired by memetic algorithms, which accelerate evolutionary search through agentic refinement, we introduce Neuro-Tangled Program Graphs. This biologically grounded extension utilizes hierarchical plasticity within the structure of an agent, applying a homeostatic rule at the initial decision edges and a competitive Oja-style update in each subsequent decision edge. Evaluated on both a static and dynamic variant of the MuJoCo Ant environment, this approach yields higher peak returns and evolves with 59-88% fewer mean effective instructions used per step, demonstrating stronger performance and a more compact search. Next, we add an TD-style online value baseline and eligibility traces to stabilize and distribute dense step-wise rewards over time, sharpening temporal updates within each agent. We then examine how trace length and a per-team plasticity decay factor shape learning dynamics. To set these, we compare end-to-end evolutionary tuning with MAP-Elites using a multi-archive that explores (trace length x decay). The benefits of reward modulation are then tested for with TPG and NeuroTPG variants on a customized static and dynamic maze environment. This addition show a consistently better performance across all seeds and also a more interpretable final structure. Overall, our findings highlight the vital role of a local search within population search algorithms. Our studies hope to open a new avenue to gradient-free memetic algorithms which offer many benefits and opportunities from various already developed field of studies."],"dc:description.degree":["Master of Science (MSc)"],"dc:identifier.uri":["http://hdl.handle.net/11375/32597","https://doi.org/10.71548/21070"],"dc:language.iso":["en"],"dc:subject":["Genetic Programming","Artificial Intelligence","Evolutionary Computation","Digital Evolution"],"dc:title":["Neuroplasticity in Genetic Programming Agents for Adaptive and Continual Decision Making"],"dc:type":["Thesis"],"others:access_status":["open.access"]},"updated_at":"2026-11-18T19:38:29Z"}