{"id":{"repo_id":"mtu","oai_identifier":"oai:digitalcommons.mtu.edu:etdr-2219"},"canonical_url":"https://search.ndltd.org/etd/mtu/oai:digitalcommons.mtu.edu:etdr-2219","repository":{"repo_id":"mtu","name":"Michigan Technological University","base_url":"https://digitalcommons.mtu.edu/do/oai/"},"display":{"title":"Proximal Reliability Optimization for Reinforcement Learning","abstract":"Model-free reinforcement learning based methods such as Proximal Policy Optimization, or Q-learning typically require thousands of interactions with the environment to approximate the optimum controller which may not always be feasible in robotics due to safety and time consumption. Model-based methods such as PILCO or BlackDrops, while data-efficient, provide solutions with limited robustness and complexity. To address this tradeoff, we introduce two distinct methods for creation of virtual environments, which are formed through limited trials conducted in the original environment. We provide an efficient method for uncertainty management, which is used as a metric for self-improvement by identification of the points with maximum expected improvement through adaptive sampling. Capturing the uncertainty also allows for better modeling the reward responses of the original system. The reliance on absolute or deterministic reward as a metric for optimization process renders reinforcement learning highly susceptible to changes in problem dynamics. We introduce a novel framework that effectively quantify the aleatory variability in the design space and induces robustnessin controllers by switching to a reliability-based optimization routine. Our approaches enable the use of complex policy structures and reward functions through a unique combination of model-based and model-free methods, while still retaining the data efficiency. We demonstrate the validity of our methods on several reinforcement learning problems in OpenAI gym. We prove that our approach offers a better modeling capacity for complex system dynamics as compared to established methods.","abstract_html":"Model-free reinforcement learning based methods such as Proximal Policy Optimization, or Q-learning typically require thousands of interactions with the environment to approximate the optimum controller which may not always be feasible in robotics due to safety and time consumption. Model-based methods such as PILCO or BlackDrops, while data-efficient, provide solutions with limited robustness and complexity. To address this tradeoff, we introduce two distinct methods for creation of virtual environments, which are formed through limited trials conducted in the original environment. We provide an efficient method for uncertainty management, which is used as a metric for self-improvement by identification of the points with maximum expected improvement through adaptive sampling. Capturing the uncertainty also allows for better modeling the reward responses of the original system. The reliance on absolute or deterministic reward as a metric for optimization process renders reinforcement learning highly susceptible to changes in problem dynamics. We introduce a novel framework that effectively quantify the aleatory variability in the design space and induces robustnessin controllers by switching to a reliability-based optimization routine. Our approaches enable the use of complex policy structures and reward functions through a unique combination of model-based and model-free methods, while still retaining the data efficiency. We demonstrate the validity of our methods on several reinforcement learning problems in OpenAI gym. We prove that our approach offers a better modeling capacity for complex system dynamics as compared to established methods.","abstract_has_math":false,"creators":["Patwardhan, Narendra"],"institution":null,"degree_name":"Master of Science in Mechanical Engineering (MS)","degree_level":"Campus Access Master's Thesis","degree_discipline":"Department of Mechanical Engineering-Engineering Mechanics","degree_department":null,"school":null,"contributors":["Zequn Wang"],"advisors":[],"committee_chairs":[],"committee_members":["Bo Chen","Ye Sun"],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T08:18:10Z","subjects":["Robotics","Uncertainty","Electro-Mechanical Systems"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.mtu.edu/etdr/1026"],"render_values":[{"text":"https://digitalcommons.mtu.edu/etdr/1026","href":"https://digitalcommons.mtu.edu/etdr/1026","code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://doi.org/10.37099/mtu.dc.etdr/1026","outbound_label":"DOI","outbound_source":"dc:identifier","record_url":"https://doi.org/10.37099/mtu.dc.etdr/1026","record_label":"Repository record","record_kind":"doi","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.mtu.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.mtu.edu%3Aetdr-2219","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Zequn Wang"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Bo Chen","Ye Sun"]},{"key":"dc:creator","label":"Author","values":["Patwardhan, Narendra"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.created","label":"Dc Date Created","values":["2020-01-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Department of Mechanical Engineering-Engineering Mechanics"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Campus Access Master's Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science in Mechanical Engineering (MS)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Robotics","Uncertainty","Electro-Mechanical Systems"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["10.37099/mtu.dc.etdr/1026","https://digitalcommons.mtu.edu/etdr/1026"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Model-free reinforcement learning based methods such as Proximal Policy Optimization, or Q-learning typically require thousands of interactions with the environment to approximate the optimum controller which may not always be feasible in robotics due to safety and time consumption. Model-based methods such as PILCO or BlackDrops, while data-efficient, provide solutions with limited robustness and complexity. To address this tradeoff, we introduce two distinct methods for creation of virtual environments, which are formed through limited trials conducted in the original environment. We provide an efficient method for uncertainty management, which is used as a metric for self-improvement by identification of the points with maximum expected improvement through adaptive sampling. Capturing the uncertainty also allows for better modeling the reward responses of the original system. The reliance on absolute or deterministic reward as a metric for optimization process renders reinforcement learning highly susceptible to changes in problem dynamics. We introduce a novel framework that effectively quantify the aleatory variability in the design space and induces robustnessin controllers by switching to a reliability-based optimization routine. Our approaches enable the use of complex policy structures and reward functions through a unique combination of model-based and model-free methods, while still retaining the data efficiency. We demonstrate the validity of our methods on several reinforcement learning problems in OpenAI gym. We prove that our approach offers a better modeling capacity for complex system dynamics as compared to established methods."]},{"key":"dc:description.note","label":"Dc Description Note","values":["<p>Resubmitted after acceptance from my advisor.</p>"]},{"key":"dc:title","label":"Title","values":["Proximal Reliability Optimization for Reinforcement Learning"]}]}],"canonical_facts":{"dc:contributor":["Zequn Wang"],"dc:contributor.committeemember":["Bo Chen","Ye Sun"],"dc:creator":["Patwardhan, Narendra"],"dc:date.created":["2020-01-01T08:00:00Z"],"dc:description.abstract":["Model-free reinforcement learning based methods such as Proximal Policy Optimization, or Q-learning typically require thousands of interactions with the environment to approximate the optimum controller which may not always be feasible in robotics due to safety and time consumption. Model-based methods such as PILCO or BlackDrops, while data-efficient, provide solutions with limited robustness and complexity. To address this tradeoff, we introduce two distinct methods for creation of virtual environments, which are formed through limited trials conducted in the original environment. We provide an efficient method for uncertainty management, which is used as a metric for self-improvement by identification of the points with maximum expected improvement through adaptive sampling. Capturing the uncertainty also allows for better modeling the reward responses of the original system. The reliance on absolute or deterministic reward as a metric for optimization process renders reinforcement learning highly susceptible to changes in problem dynamics. We introduce a novel framework that effectively quantify the aleatory variability in the design space and induces robustnessin controllers by switching to a reliability-based optimization routine. Our approaches enable the use of complex policy structures and reward functions through a unique combination of model-based and model-free methods, while still retaining the data efficiency. We demonstrate the validity of our methods on several reinforcement learning problems in OpenAI gym. We prove that our approach offers a better modeling capacity for complex system dynamics as compared to established methods."],"dc:description.note":["<p>Resubmitted after acceptance from my advisor.</p>"],"dc:identifier":["10.37099/mtu.dc.etdr/1026","https://digitalcommons.mtu.edu/etdr/1026"],"dc:subject":["Robotics","Uncertainty","Electro-Mechanical Systems"],"dc:title":["Proximal Reliability Optimization for Reinforcement Learning"],"thesis:degree_discipline":["Department of Mechanical Engineering-Engineering Mechanics"],"thesis:degree_level":["Campus Access Master's Thesis"],"thesis:degree_name":["Master of Science in Mechanical Engineering (MS)"]},"updated_at":"2026-11-19T08:18:10Z"}