Back to search

University of Washington

Language Models can Generalize from Indirect Evidence: Evidence from Filtered Corpus Training (FICT)

Abstract

dc:description.abstract

This thesis introduces Filtered Corpus Training, a method that trains language models (LMs) on corpora with certain linguistic constructions filtered out from the training data, and uses it to measure the ability of LMs to perform linguistic generalization on the basis of indirect evidence. Applying the method to both LSTM and Transformer LMs, of roughly comparable size, we develop corpora filtered of direct evidence for a wide range of linguistic phenomena. Our results show that while transformers are better qua LMs (as measured by perplexity), both models perform equally and surprisingly well on linguistic generalization measures, suggesting that they are capable of generalizing from indirect evidence. This adds to a growing body of evidence on the limitations of perplexity as an evaluation metric, while also showing that direct attestation may be not strictly be necessary for learners to develop the appropriate linguistic generalizations.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Patil, Abhinav
Advisor dc:contributor.advisor
  • Steinert-Threlkeld, Shane

Subjects

dc:subject × 9

Rights

dc:rights
Statement dc:rights
  • CC BY-SA
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1773/52074

Chain of custody

source
Harvested from
University of Washington
Base URL
digital.lib.washington.edu/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Patil, Abhinav. Language Models can Generalize from Indirect Evidence: Evidence from Filtered Corpus Training (FICT). 2024. https://hdl.handle.net/1773/52074