Back to search

Georgia Institute of Technology

Entropic-Regularized Second-Order Dynamic Optimization

Abstract

dc:description.abstract

This dissertation addresses the challenge of executing optimal control for dynamical systems operating in highly non-convex landscapes arising from nonlinear dynamics and cluttered environments. While classical second-order trajectory optimization methods such as Differential Dynamic Programming (DDP) offer high computational efficiency and fast local convergence, they are vulnerable to getting trapped in poor local minima. Conversely, pure sampling-based methods can bypass local minima but often suffer from noisy decision-making, high sample complexity, and sensitivity to sampling covariances that require tuning. To resolve these problems, this work explores the framework of Maximum Entropy DDP (ME-DDP), where an exploration covariance arises naturally from its derivation via entropic regularization. We formalize a universal control scheme that leverages gradient steps for local exploitation while periodically perturbing the trajectory to ensure exploration. We extend this mechanism beyond standard Shannon entropy to more generalized representations, presenting the development of Tsallis entropic regularization and Stein Variational DDP (SV-DDP) to maintain policy diversity. The efficacy of the proposed algorithms is verified by benchmarking against the most successful sampling-based method, the variants of Model Predictive Path Integral (MPPI) control, in a Model Predictive Control (MPC) framework, using robotic navigation tasks in cluttered environments. Our analysis demonstrates that while pure sampling-based methods can discover behaviors that ME-DDP variants miss, ME-DDP variants yield superior overall performance. Finally, the practical viability and robustness of this framework are validated through hardware experiments on a quadrotor navigating cluttered environments.

Degree

thesis:*
Name thesis:degree_name
Machine Learning, PhD
Department thesis:degree_department
Aerospace Engineering
Grantor
Georgia Institute of Technology
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Aoyama, Yuichiro
Advisor dc:contributor.advisor
  • Theodorou, Evangelos A.
Committee members dc:contributor.committeemember
  • Chou, Glen
  • Vamvoudakis, Kyriakos
  • Dellaert, Frank
  • Oguri, Kenshiro

Rights

Language dc:language.iso
English

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/82116

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Aoyama, Yuichiro. Entropic-Regularized Second-Order Dynamic Optimization. Georgia Institute of Technology, 2026. https://hdl.handle.net/1853/82116