Back to search

Cornell University

Robust and Scalable Spectral Topic Modeling for Large Vocabularies

Abstract

dc:description.abstract

Across many data domains, co-occurrence statistics about the joint appearance of objects are powerfully informative. In topic modeling, spectral methods can provably learn low-dimensional latent topics from easily-collected word co-occurrence statistics unlike likelihood-based methods that require exhaustive reiterations through the corpus. However, spectral methods suffer from two major drawbacks: the quality of learned topics deteriorates drastically when the empirical data does not follow the generative model, and the co-occurrence statistics itself grows to an intractable size when working with large vocabularies. This thesis is an attempt to overcome these drawbacks by developing a scalable and robust spectral topic inference framework based on Joint Stochastic Matrix Factorization. First, we provide theoretical foundations of spectral topic inference as well as step-wise algorithmic implementations of our anchor-based approach that can learn quality topics despite model-data mismatch. We then scale towards larger vocabularies by operating solely on compressed low-rank representations of co-occurrence statistics, keeping the overall cost linear with respect to the vocabulary size. Quantitative and qualitative experiments on various datasets not only demonstrate our framework's consistency and efficiency in inferring high-quality topics, but also introduce improvements in interpretability of the individual topics.

Degree

thesis:*
Name thesis:degree_name
M.S., Computer Science
Level thesis:degree_level
Master of Science
Discipline thesis:degree_discipline
Computer Science
Grantor
Cornell University
Year dc:date.issued
2020

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Cho, Sungjun
Chair dc:contributor.chair
  • Bindel, David
Committee member dc:contributor.committeemember
  • Mimno, David

Subjects

dc:subject × 4

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
13254524
Dc Identifier Other
ProQuest Submission ID: 10866
ProQuest Publication ID: 27962590

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Cho, Sungjun. Robust and Scalable Spectral Topic Modeling for Large Vocabularies. Master of Science thesis, Cornell University, 2020. https://hdl.handle.net/1813/70307