Back to search

Cornell University

Structure in Prediction: Mapping Large Language Model Prediction to Symbolic Structure

Abstract

dc:description.abstract

Modern large language models exhibit immense capability, yet their internal predictive processes remain largely opaque. They are fundamentally sequence modelers, assigning probabilities to the surface-form representations of underlying systems like logic, code, and language, that reflect underlying systems with their own rigid constraints and structural requirements. When the predictive process is treated as agnostic to these formal properties, we are left with a model that may solve complex problems, but we remain uncertain whether it did so via a robust internal logic or merely through probabilistic luck. This thesis proposes that aligning a model's latent processing to a task's formal requirements governs effective neural reasoning. We demonstrate that by aligning the various levels of the language modeling stack with symbolic structures, we can move from black-box emergence toward informed intervention and design. The first chapter addresses the challenge of verifiable search in zero-tolerance domains. We present a neurosymbolic framework that aligns internal neural artifacts, such as attention maps and prediction entropy, with formal symbolic specifications. By using these artifacts to identify uncertain neural guesses and lift them into symbolic solvers, we achieve mathematically guaranteed correctness in sensitive program translation tasks. This approach leverages the model as a flexible generator while anchoring its output in the formal semantics of the target machine language.The second chapter revisits classical computational complexity theory through the lens of modern inference. By modeling reasoning tasks as their underlying Deterministic Finite Automata (DFAs), we demonstrate that a task’s intrinsic state-space complexity effectively predicts the test-time compute required for a reliably correct solution. This allows for principled compute budgeting, moving beyond heuristic prompting toward a rigorous theory of inference-time scaling. The third chapter investigates the scaling efficiency of atomic units under different prediction objectives. We quantify the computational cost incurred when the linguistic and structural priors of subword tokenization and causal order are removed. Through a compute-matched study of byte-level modeling, we show that learning efficiency is not a property of scale alone, but a function of the alignment between the model’s objective (autoregressive vs. masked diffusion) and the sequential nature of the data. Together, these studies support a unifying principle: whether provided by the data representation, the training objective, or an external prior, structure governs the efficiency and reliability of neural reasoning. This thesis identifies key opportunities to inform and improve model inference by aligning neural predictive processes with task structure.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Computer Science
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Computer Science
Grantor
Cornell University
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lee, Celine
Chair dc:contributor.chair
  • Rush, Alexander
Committee members dc:contributor.committeemember
  • van Schijndel, Marten
  • Ellis, Kevin

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Other
Lee_cornellgrad_0058F_15576
https://www.proquest.com/LegacyDocView/DISSNUM/32676441

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Lee, Celine. Structure in Prediction: Mapping Large Language Model Prediction to Symbolic Structure. Doctor of Philosophy thesis, Cornell University, 2026. https://hdl.handle.net/1813/126574