Back to search

Cornell University

Stochastic optimal control: threshold-aware policies and impact of random disruptions

Abstract

dc:description.abstract

Stochastic optimal control theory encompasses various types of stochasticity and notions of optimality. The standard risk-neutral approach minimizes or maximizes an expected total cost, but this approach often yields non-robust results. In this thesis, we introduce a particular type of robust control framework of indefinite-horizon processes, maximizing the probability of desired outcomes while keeping the cumulative cost within a threshold. For diffusive processes, our framework results in second-order parabolic Hamilton-Jacobi-Bellman (HJB) Partial Differential Equations (PDEs).We develop an efficient algorithm to solve these equations by leveraging the inherent causality of the framework. This allows us to recover the optimal "threshold (risk)-aware" feedback policies for all initial configurations and a range of threshold values simultaneously in a single sweep. We first apply this methodology to adaptive cancer therapy under stochastic cancer dynamics. In particular, we aim to maximize the probability of achieving treatment goals while keeping the total treatment cost within a specific cost threshold/budget. We then extend this threshold-aware approach to hybrid control problems, specifically through sailboat routing under stochastically evolving wind conditions. This application involves solving a pair of quasi-variational inequalities in a Hamilton-Jacobi framework. Monte Carlo simulations are used to generate cumulative distribution functions (CDFs), demonstrating the advantages of threshold-aware policies over risk-neutral ones. In the final section, we investigate bacterial competition influenced by environmental extreme events (dilutions). We propose an explanation for why toxin-sensitive bacteria, usually outcompeted by toxin-producers in vitro, can thrive under frequent dilutions. We consider both deterministic periodic dilutions and randomly timed dilutions modeled by a Poisson process. Through a series of optimized toxin-regulation behaviors for toxin-producers, we demonstrate that toxin-sensitive strains still have a reasonable chance of winning. The numerical approach involves solving Hamilton-Jacobi-type equations (including a specific type of non-local coupling emerging from the jump-discontinuities induced by the Poisson process) using semi-Lagrangian schemes.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Applied Mathematics
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Applied Mathematics
Grantor
Cornell University
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Wang, MingYi
Chair dc:contributor.chair
  • Vladimirsky, Alexander
Committee members dc:contributor.committeemember
  • Sosoe, Philippe
  • Damle, Anil

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
16611914
Dc Identifier Other
ProQuest Submission ID: 14591
ProQuest Publication ID: 31488853

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Wang, MingYi. Stochastic optimal control: threshold-aware policies and impact of random disruptions. Doctor of Philosophy thesis, Cornell University, 2024. https://hdl.handle.net/1813/116611