Back to search

University of British Columbia

Robust reinforcement learning with intrinsic stochasticity in real-time simulation

Abstract

dc:description

With the advent of Industry 4.0, we see a push for manufacturing processes to be more intelligent. One of the main components of a manufacturing process are the robots, which also must be intelligent and more autonomous. One method of achieving intelligent robots is the use of reinforcement learning (RL). However, the need for large data samples for learning forces us to use simulations to train RL agents for robotic tasks. While simulations help alleviate the data burden, they can negatively impact the robustness of RL agents. Since simulations are not necessarily accurate representations of the real-world, the agents trained in these environments are sensitive to modelling errors and input disturbances, resulting in failure when transferred to the real-world. Many solutions exist to improve the sim-to-real transfer of agents to make them more robust. These solutions include system identification, domain adaptation, and domain randomization. While these methods succeed in creating more robust agents, they are highly extensive, heuristic, and require expert knowledge to implement them. In this work, we propose a method based on the principles of domain randomization that produces robust agents with simpler implementation. We start by proposing a novel simulation platform that incorporates robot dynamics into an existing industrial simulation tool, CIROS, to allow high-fidelity simulations of manufacturing processes. During the testing of this platform, we notice some intrinsic stochasticity in the robot trajectories caused by its real-time nature. We investigate this intrinsic stochasticity further and find that it is comparable to the stochasticity of a real-robot. Based on this finding, we propose using the intrinsic stochasticity as a simpler alternative to domain randomization for robust RL. We validate this claim by training and evaluating the robustness of an agent trained on the intrinsic stochasticity. We found that our method produces a significant level of robustness and can indeed be a viable alternative that is easier to implement than existing solutions.

Degree

thesis:*
Name thesis:degree_name
Master of Applied Science - MASc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Mechanical Engineering
Grantor dc:publisher
University of British Columbia
Year dc:date
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Dershan, Ram
Contributors dc:contributor
  • Richert, Dean
  • Najjaran, Homayoun

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/82345

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Dershan, Ram. Robust reinforcement learning with intrinsic stochasticity in real-time simulation. master's thesis, University of British Columbia, 2022. http://hdl.handle.net/2429/82345