Back to search

University of British Columbia

Physics-based character controllers with reinforcement learning

Abstract

dc:description

Physics-based character control has advanced significantly in recent years, with reinforcement learning (RL) emerging as a powerful method for producing general, versatile controllers. However, applying RL to humanoid control in animation and robotics poses fundamental challenges, including brittle policies, difficulties in exploring high-dimensional spaces, and a reliance on high-quality motion capture data. This thesis addresses these challenges, offering insights and methods in physics-based character control. We begin with a survey of learning methods for humanoid control, identifying the exploration problem as a fundamental limitation for developing general controllers. The absence of expert data and the complexity of state-action spaces hinder RL’s effectiveness. We review existing solutions to these challenges, highlighting their strengths and limitations. Next, we show the crucial role of environment design in RL. Poor design choices in key components, such as state representations, reward structures, or action spaces, can result in brittle and inefficient learning. Our analysis highlights how thoughtful environment enhances controller robustness and generalization. A key challenge in RL is the heavy reliance on extensive, high-quality motion capture datasets to guide learning. This raises the question: what happens if we lack data for specific motions, either because it is unavailable or unfeasible to collect? To address this, we introduce two strategies that reduce the dependency on motion capture data. The first approach uses a simplified physical model to provide motion priors, enabling controllers to learn complex behaviors without reference data. Through learning a brachiating controller, we show how the simplified model guides the center-of-mass trajectory and grasp timing, allowing the full model to efficiently learn swinging behaviors. The second strategy focuses on sparse motion retargeting, adapting data from unrelated characters to new morphologies. Our framework retargets sparse sensor data to physically simulated characters with different skeletal structures. Using physics as a prior to refine the kinematic motion and overcoming the exploration problem, our method produces robust real-time controllers for applications in virtual reality (VR) and robotics. Together, these contributions enable the development of robust, adaptable controllers, broadening RL’s applicability to physics-based animations and advancing the state of the art in learning character control.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy - PhD
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Reda, Daniele
Contributors dc:contributor
  • Van de Panne, M. (Michiel), 1965-

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/90456

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Reda, Daniele. Physics-based character controllers with reinforcement learning. doctoral thesis, University of British Columbia, 2025. http://hdl.handle.net/2429/90456