Back to search

Cornell University

Memory in Language Models: Representation and Extraction

Abstract

dc:description.abstract

We explore memory in neural language models as it is stored in model weights after training and in model activations ("embeddings") during inference. We first describe a method for improving embeddings by incorporating surrounding documents in the context of text retrieval. We then propose a new method for inverting text embeddings and demonstrate its applicability to the outputs of general language models. Finally, we measurement of the information content of model weights, a way to characterize the total amount of information models can store.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Computer Science
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Computer Science
Grantor
Cornell University
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Morris, John
Chair dc:contributor.chair
  • Rush, Alexander
Committee members dc:contributor.committeemember
  • Zabih, Ramin
  • Pierson, Emma

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
17423278
Dc Identifier Other
ProQuest Submission ID: 15353
ProQuest Publication ID: 32286145

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Morris, John. Memory in Language Models: Representation and Extraction. Doctor of Philosophy thesis, Cornell University, 2025. https://hdl.handle.net/1813/121141