Georgia Institute of Technology
Entropic-Regularized Second-Order Dynamic Optimization
Abstract
dc:description.abstractThis dissertation addresses the challenge of executing optimal control for dynamical systems operating in highly non-convex landscapes arising from nonlinear dynamics and cluttered environments. While classical second-order trajectory optimization methods such as Differential Dynamic Programming (DDP) offer high computational efficiency and fast local convergence, they are vulnerable to getting trapped in poor local minima. Conversely, pure sampling-based methods can bypass local minima but often suffer from noisy decision-making, high sample complexity, and sensitivity to sampling covariances that require tuning. To resolve these problems, this work explores the framework of Maximum Entropy DDP (ME-DDP), where an exploration covariance arises naturally from its derivation via entropic regularization. We formalize a universal control scheme that leverages gradient steps for local exploitation while periodically perturbing the trajectory to ensure exploration. We extend this mechanism beyond standard Shannon entropy to more generalized representations, presenting the development of Tsallis entropic regularization and Stein Variational DDP (SV-DDP) to maintain policy diversity. The efficacy of the proposed algorithms is verified by benchmarking against the most successful sampling-based method, the variants of Model Predictive Path Integral (MPPI) control, in a Model Predictive Control (MPC) framework, using robotic navigation tasks in cluttered environments. Our analysis demonstrates that while pure sampling-based methods can discover behaviors that ME-DDP variants miss, ME-DDP variants yield superior overall performance. Finally, the practical viability and robustness of this framework are validated through hardware experiments on a quadrotor navigating cluttered environments.
Degree
thesis:*- Name thesis:degree_name
- Machine Learning, PhD
- Department thesis:degree_department
- Aerospace Engineering
- Grantor
- Georgia Institute of Technology
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Aoyama, Yuichiro
- Advisor dc:contributor.advisor
-
- Theodorou, Evangelos A.
- Committee members dc:contributor.committeemember
-
- Chou, Glen
- Vamvoudakis, Kyriakos
- Dellaert, Frank
- Oguri, Kenshiro
Rights
- Language dc:language.iso
- English
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/82116