{"id":{"repo_id":"ets-quebec","oai_identifier":"oai:espace.etsmtl.ca:3735"},"canonical_url":"https://search.ndltd.org/etd/ets-quebec/oai:espace.etsmtl.ca:3735","repository":{"repo_id":"ets-quebec","name":"ETS (Quebec)","base_url":"https://espace.etsmtl.ca/cgi/oai2"},"display":{"title":"Efficient reinforcement learning using improved prior modeling","abstract":"Reinforcement learning (RL) has emerged as a unifying framework for sequential decision making. Yet, its practical impact is curbed by three persistent limitations: prohibitive sample demands, poor generalization across tasks and domains, and a lack of interpretability that accompanies high-capacity function approximators. This research argues that these limitations share a common source-insufficient prior structure—and that they can be alleviated by modelling agents with learnable, discrete, and adaptive priors. We pursue this claim through a trilogy of works that together reshape the reward, the world model, and the skill library on which efficient learning depends. The first work converts sparse, human-supplied scores for heavy-equipment operation into a dense, differentiable reward prior. By training a score predictor that both evaluates and guides the policy, we demonstrate safe exploration in environment interaction relative to hand-crafted heuristics. The second work introduces DART, a transformer-based architecture that tokenizes states, actions, and returns into a standard symbolic alphabet. This discrete world model captures long-range temporal dependencies while preserving the manipulability of tokens, delivering state-of-the-art sample efficiency on the Atari-100k benchmark without sacrificing final performance. The third work presents STRIDE, a dynamic vector-quantized VAE that autonomously expands a codebook of motor primitives as tasks accumulate. STRIDE retains previously learned behaviours with negligible degradation. It halves the adaptation time on challenging locomotion curricula, all while exposing an interpretable “skill trace” that allows practitioners to audit and debug decisions. Collectively, these contributions substantiate a unified hypothesis: when knowledge about rewards, dynamics, and skills is cast into discrete, growing vocabularies, tabula-rasa search gives way to data-efficient, compositional, and transparent learning. Beyond empirical gains—up to a 2.6× speed-up across diverse domains—the work offers conceptual tools for reasoning about priors in RL, provides open-source implementations for community use, and outlines future directions toward multi-modal and human-editable prior structures. In doing so, it takes a decisive step toward RL systems that can be trusted to learn quickly, allowing transferability and explainability when it matters most.","abstract_html":"Reinforcement learning (RL) has emerged as a unifying framework for sequential decision making. Yet, its practical impact is curbed by three persistent limitations: prohibitive sample demands, poor generalization across tasks and domains, and a lack of interpretability that accompanies high-capacity function approximators. This research argues that these limitations share a common source-insufficient prior structure—and that they can be alleviated by modelling agents with learnable, discrete, and adaptive priors. We pursue this claim through a trilogy of works that together reshape the reward, the world model, and the skill library on which efficient learning depends. The first work converts sparse, human-supplied scores for heavy-equipment operation into a dense, differentiable reward prior. By training a score predictor that both evaluates and guides the policy, we demonstrate safe exploration in environment interaction relative to hand-crafted heuristics. The second work introduces DART, a transformer-based architecture that tokenizes states, actions, and returns into a standard symbolic alphabet. This discrete world model captures long-range temporal dependencies while preserving the manipulability of tokens, delivering state-of-the-art sample efficiency on the Atari-100k benchmark without sacrificing final performance. The third work presents STRIDE, a dynamic vector-quantized VAE that autonomously expands a codebook of motor primitives as tasks accumulate. STRIDE retains previously learned behaviours with negligible degradation. It halves the adaptation time on challenging locomotion curricula, all while exposing an interpretable “skill trace” that allows practitioners to audit and debug decisions. Collectively, these contributions substantiate a unified hypothesis: when knowledge about rewards, dynamics, and skills is cast into discrete, growing vocabularies, tabula-rasa search gives way to data-efficient, compositional, and transparent learning. Beyond empirical gains—up to a 2.6× speed-up across diverse domains—the work offers conceptual tools for reasoning about priors in RL, provides open-source implementations for community use, and outlines future directions toward multi-modal and human-editable prior structures. In doing so, it takes a decisive step toward RL systems that can be trusted to learn quickly, allowing transferability and explainability when it matters most.","abstract_has_math":false,"creators":["Agarwal, Pranav"],"institution":"École de technologie supérieure","degree_name":"Thèse de doctorat électronique","degree_level":"phd","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-09-24","date_published":"2025-09-24","updated_at":"2026-11-18T22:10:50Z","subjects":["apprentissage par renforcement efficace, modélisation des a priori, représentations discrètes, apprentissage continu de compétences, interprétabilité"],"languages":["en"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["INSERT-YOUR-LIBRARY-ID-HERE3735"],"render_values":[{"text":"INSERT-YOUR-LIBRARY-ID-HERE3735","href":null,"code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":null,"outbound_label":null,"outbound_source":null,"record_url":null,"record_label":null,"record_kind":null,"record_source":null,"download_url":"https://espace.etsmtl.ca/id/eprint/3735/1/Agarwal_Pranav.pdf","download_label":"Download PDF","download_source":"dc:identifier","access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://espace.etsmtl.ca/cgi/oai2?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Aespace.etsmtl.ca%3A3735","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:creator","label":"Author","values":["Agarwal, Pranav"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-09-24"]},{"key":"dc:publisher","label":"Institution","values":["École de technologie supérieure"]},{"key":"dc:type","label":"Dc Type","values":["Electronic Thesis or Dissertation"]},{"key":"thesis:degree_level","label":"Degree Level","values":["phd"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Thèse de doctorat électronique"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["École de technologie supérieure"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["apprentissage par renforcement efficace, modélisation des a priori, représentations discrètes, apprentissage continu de compétences, interprétabilité"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["INSERT-YOUR-LIBRARY-ID-HERE3735","https://espace.etsmtl.ca/id/eprint/3735/1/Agarwal_Pranav.pdf"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Reinforcement learning (RL) has emerged as a unifying framework for sequential decision making. Yet, its practical impact is curbed by three persistent limitations: prohibitive sample demands, poor generalization across tasks and domains, and a lack of interpretability that accompanies high-capacity function approximators. This research argues that these limitations share a common source-insufficient prior structure—and that they can be alleviated by modelling agents with learnable, discrete, and adaptive priors. We pursue this claim through a trilogy of works that together reshape the reward, the world model, and the skill library on which efficient learning depends. The first work converts sparse, human-supplied scores for heavy-equipment operation into a dense, differentiable reward prior. By training a score predictor that both evaluates and guides the policy, we demonstrate safe exploration in environment interaction relative to hand-crafted heuristics. The second work introduces DART, a transformer-based architecture that tokenizes states, actions, and returns into a standard symbolic alphabet. This discrete world model captures long-range temporal dependencies while preserving the manipulability of tokens, delivering state-of-the-art sample efficiency on the Atari-100k benchmark without sacrificing final performance. The third work presents STRIDE, a dynamic vector-quantized VAE that autonomously expands a codebook of motor primitives as tasks accumulate. STRIDE retains previously learned behaviours with negligible degradation. It halves the adaptation time on challenging locomotion curricula, all while exposing an interpretable “skill trace” that allows practitioners to audit and debug decisions. Collectively, these contributions substantiate a unified hypothesis: when knowledge about rewards, dynamics, and skills is cast into discrete, growing vocabularies, tabula-rasa search gives way to data-efficient, compositional, and transparent learning. Beyond empirical gains—up to a 2.6× speed-up across diverse domains—the work offers conceptual tools for reasoning about priors in RL, provides open-source implementations for community use, and outlines future directions toward multi-modal and human-editable prior structures. In doing so, it takes a decisive step toward RL systems that can be trusted to learn quickly, allowing transferability and explainability when it matters most."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Efficient reinforcement learning using improved prior modeling"]}]}],"canonical_facts":{"dc:creator":["Agarwal, Pranav"],"dc:date":["2025-09-24"],"dc:description":["Reinforcement learning (RL) has emerged as a unifying framework for sequential decision making. Yet, its practical impact is curbed by three persistent limitations: prohibitive sample demands, poor generalization across tasks and domains, and a lack of interpretability that accompanies high-capacity function approximators. This research argues that these limitations share a common source-insufficient prior structure—and that they can be alleviated by modelling agents with learnable, discrete, and adaptive priors. We pursue this claim through a trilogy of works that together reshape the reward, the world model, and the skill library on which efficient learning depends. The first work converts sparse, human-supplied scores for heavy-equipment operation into a dense, differentiable reward prior. By training a score predictor that both evaluates and guides the policy, we demonstrate safe exploration in environment interaction relative to hand-crafted heuristics. The second work introduces DART, a transformer-based architecture that tokenizes states, actions, and returns into a standard symbolic alphabet. This discrete world model captures long-range temporal dependencies while preserving the manipulability of tokens, delivering state-of-the-art sample efficiency on the Atari-100k benchmark without sacrificing final performance. The third work presents STRIDE, a dynamic vector-quantized VAE that autonomously expands a codebook of motor primitives as tasks accumulate. STRIDE retains previously learned behaviours with negligible degradation. It halves the adaptation time on challenging locomotion curricula, all while exposing an interpretable “skill trace” that allows practitioners to audit and debug decisions. Collectively, these contributions substantiate a unified hypothesis: when knowledge about rewards, dynamics, and skills is cast into discrete, growing vocabularies, tabula-rasa search gives way to data-efficient, compositional, and transparent learning. Beyond empirical gains—up to a 2.6× speed-up across diverse domains—the work offers conceptual tools for reasoning about priors in RL, provides open-source implementations for community use, and outlines future directions toward multi-modal and human-editable prior structures. In doing so, it takes a decisive step toward RL systems that can be trusted to learn quickly, allowing transferability and explainability when it matters most."],"dc:format":["application/pdf"],"dc:identifier":["INSERT-YOUR-LIBRARY-ID-HERE3735","https://espace.etsmtl.ca/id/eprint/3735/1/Agarwal_Pranav.pdf"],"dc:language":["en"],"dc:publisher":["École de technologie supérieure"],"dc:subject":["apprentissage par renforcement efficace, modélisation des a priori, représentations discrètes, apprentissage continu de compétences, interprétabilité"],"dc:title":["Efficient reinforcement learning using improved prior modeling"],"dc:type":["Electronic Thesis or Dissertation"],"thesis:degree_level":["phd"],"thesis:degree_name":["Thèse de doctorat électronique"],"thesis:institution_name":["École de technologie supérieure"]},"updated_at":"2026-11-18T22:10:50Z"}