Back to search

Georgia Institute of Technology

Scalable and Safe Deep Learning Architectures for Stochastic Optimal Control Using Forward-Backward Stochastic Differential Equations

Abstract

dc:description.abstract

Stochastic Optimal Control (SOC) in continuous-time requires solving the Hamilton-Jacobi-Bellman (HJB) equation which suffers from the well-known curse-of-dimensionality. Instead of directly attempting to solve the HJB, one can obtain probabilistic representations of the solution via the Nonlinear Feynman-Kac lemma which relates the unique solution of the HJB to a system of Forward-Backward Stochastic Differential Equations (FBSDEs). This thesis develops novel algorithms that leverage the function approximation capabilities of deep recurrent neural networks to solve systems of FBSDEs and the resulting deep FBSDE framework is memory efficient, provides temporally smoother controls, is immune to compounding approximation errors and can be employed for long time-horizons owing to the underlying Long-Short Term Memory network architecture. Starting from a Vanilla SOC problem, the framework is extended to problem formulations such as dynamics with control-multiplicative noise, dynamics with non-affine controls and non-quadratic control cost functions, safety-critical tasks which employs Stochastic Control Barrier Functions and for L1-SOC in minimum-fuel aerospace applications. Each problem formulation is accompanied with necessary structural modifications to the deep learning architecture to enable end-to-end learning. In order to improve the scalability of the framework, especially for safety critical multi-robot problems, this thesis then proposes a novel Decentralized Approach to Safe SOC using the aforementioned Deep FBSDE framework and the well-known Alternating Direction Method of Multipliers (ADMM) algorithm. Using simulations, the efficacy of the decentralized approach is demonstrated on challenging tasks involving many robots and safety constraints. Finally, using this as the backbone, a novel sim2real approach is developed which empowers the deep FBSDE framework to be directly deployed on hardware after training in simulation and the approach is tested on the Robotarium platform. This marks the first work to deploy FBSDE-based controllers on real hardware.

Degree

thesis:*
Level thesis:degree_level
Doctoral
Department dc:contributor.department
Aerospace Engineering
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Pereira, Marcus Aloysius
Advisor dc:contributor.advisor
  • Theodorou, Evangelos A.
Committee members dc:contributor.committeemember
  • Zhou, Enlu
  • Coogan, Samuel
  • Vamvoudakis, Kyriakos G.
  • Chen, Yongxin
  • Exarchos, Ioannis

Subjects

dc:subject × 5

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/72999

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Pereira, Marcus Aloysius. Scalable and Safe Deep Learning Architectures for Stochastic Optimal Control Using Forward-Backward Stochastic Differential Equations. Doctoral thesis, Georgia Institute of Technology, 2022. https://hdl.handle.net/1853/72999