Abstract
dc:description.abstractWe explore memory in neural language models as it is stored in model weights after training and in model activations ("embeddings") during inference. We first describe a method for improving embeddings by incorporating surrounding documents in the context of text retrieval. We then propose a new method for inverting text embeddings and demonstrate its applicability to the outputs of general language models. Finally, we measurement of the information content of model weights, a way to characterize the total amount of information models can store.
Degree
thesis:*- Name thesis:degree_name
- Ph. D., Computer Science
- Level thesis:degree_level
- Doctor of Philosophy
- Discipline thesis:degree_discipline
- Computer Science
- Grantor
- Cornell University
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Morris, John
- Chair dc:contributor.chair
-
- Rush, Alexander
- Committee members dc:contributor.committeemember
-
- Zabih, Ramin
- Pierson, Emma
Subjects
dc:subject × 6Rights
dc:rights- Statement dc:rights
-
- Attribution 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Bibid
- 17423278
- Dc Identifier Other
-
ProQuest Submission ID: 15353
ProQuest Publication ID: 32286145