Back to search

Virginia Tech

Understanding and Enhancing Sequential Decision-Making and Alignment in Foundation Models

Abstract

dc:description.abstract

Foundation models are used as sequential decision-makers, in tasks whose outputs unfold as dependent steps: reasoning, planning, decisions affecting several parties, and generation under safety requirements. This dissertation asks which factors govern the performance of foundation models in such settings and how that performance can be improved. The answer locates those factors in training data and training procedure. Pretraining text preserves finished solutions far more often than the failed attempts behind them, so a model trained on it defaults to confident, linearly coherent continuation; exploring alternative paths and retracting mistaken steps are underrepresented behaviors, not absent capabilities. The contributions elicit these behaviors in context and then teach them by supervised and reinforcement learning. One observation recurs: a model given room to explore diverse solution paths and to back out of its own mistakes makes better decisions, and in safety-critical settings the same capacity lets it recover from unsafe trajectories. The body has three parts. Part I treats foundation models in reinforcement learning: a meta-learning method for sequences of derivative-free optimization tasks, with task-averaged regret guarantees; policy optimization under several reward objectives and hard safety constraints, with a rectification step that restores feasibility after a detected violation; and an analysis tracing in-context reinforcement learning to the diversity of the pretraining task distribution. Part II treats large language models in sequential decision-making: a tool-use framework that translates natural-language energy-management requests into solver-ready optimization programs; the Algorithm of Thoughts, a prompting strategy whose exemplars record a search process so that the model explores, prunes, and backtracks within a single generation; an extension to autonomous long-horizon planning; and a training pipeline that makes the behavior a concise default. Part III turns the same capacity to alignment and safety: a prompting framework that surveys a decision's consequences for every affected stakeholder before answering, and a reinforcement-learning method that trains the backtracking step as a safety signal, so that the model retracts an emerging violation and continues from the safe prefix. Recovery complements avoidance rather than replacing it; an integrated red-team study of a static classifier defense motivates judging safety on the generated trajectory.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Computer Engineering
Department dc:contributor.department
Electrical and Computer Engineering
Grantor dc:publisher
Virginia Tech
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Sel, Bilgehan
Chair dc:contributor.committeechair
  • Jin, Ming
Committee members dc:contributor.committeemember
  • Ramakrishnan, Narendran
  • Jia, Ruoxi
  • Yanardag Delul, Pinar
  • Soysal, Alkan

Subjects

dc:subject × 5

Rights

dc:rights
Statement dc:rights
  • In Copyright
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Other
vt_gsexam:47535

Chain of custody

source
Harvested from
Virginia Tech
Base URL
vtechworks.lib.vt.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Sel, Bilgehan. Understanding and Enhancing Sequential Decision-Making and Alignment in Foundation Models. doctoral thesis, Virginia Tech, 2026. https://hdl.handle.net/10919/143769