Back to search

University of Washington

Agile Legged Robots through Reinforcement Learning and Optimal Control

Abstract

dc:description.abstract

This thesis addresses the challenge of developing agile legged robot controllers capable of high-speed, precise, and rapidly adaptive behaviors in real-world scenarios. While recent advancements have demonstrated impressive hardware capabilities of legged robots in controlled environments, deploying these controllers in complex, real-world environments remains a significant challenge. Traditional optimal control methods, which rely on predefined physics models to optimize motor commands, can precisely track desired motions but cannot plan for complex, long-horizon trajectories due to computational constraints. On the other hand, reinforcement learning frameworks offer the potential to learn versatile, perception-integrated motion policies end-to-end. However, they often lack the precision and robustness of optimal control methods and require extensive tuning in reward shaping and sim-to-real, for effective real-world application. In this thesis, we propose a hierarchical framework that merges the versatility of reinforcement learning with the precision of optimal control for enhanced agility of legged robots. We explore key challenges in training and deploying this framework and suggest methods to extend it to novel environments and tasks. We introduce the proposed hierarchical learning-control framework in three stages. First, we develop an early version of this framework for learning energy-efficient gait transitions in high-speed locomotion, with a high-level gait policy and a low-level centroidal controller. We then expand the interface between the high-level policy and the low-level controller for advanced control of continuous jumping motions, and restructure the low-level optimal control problem for GPU-accelerated training. Lastly, to achieve real-world terrain-aware jumping, we integrate perception into the framework and redesign critical components for robust real-world performance. With this final version of our framework, we achieve high-speed, animal-like jumping on challenging terrains such as stairs and stepping stones. Next, we extend the proposed hierarchical learning framework to novel terrains and tasks. By incorporating semantic information into the perception pipeline, we enable the legged robot to navigate quickly and safely through complex offroad terrains such as rocks, mud, or vegetation. Furthermore, we increased the capabilities of quadrupedal robots from basic locomotion to versatile loco-manipulation, with a novel lightweight gripper design and a restructured hierarchical framework optimized for teleoperation. This thesis delivers general-purpose algorithmic frameworks for perception-integrated, highly dynamic motion control of legged robots, as well as open-source implementations that can be readily deployed or further developed by the broader robotics community. To demonstrate their robustness and applicability, the methods proposed have been rigorously tested in standard real-robot benchmark tests and in diverse, complex real-world scenarios.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Yang, Yuxiang
Advisor dc:contributor.advisor
  • Boots, Byron

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • CC BY
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1773/52467

Chain of custody

source
Harvested from
University of Washington
Base URL
digital.lib.washington.edu/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Yang, Yuxiang. Agile Legged Robots through Reinforcement Learning and Optimal Control. 2024. https://hdl.handle.net/1773/52467