Cornell University
Text Processing for the Effective Application of Latent Dirichlet Allocation
Abstract
dc:description.abstractDistributional semantic models such as LDA (Blei et al., 2003) are a powerful method to extract patterns of word co-occurrences for exploration of a textual corpus. This is of particular interest to social scientists and humanists, who may wish to explore large collections of text in their fields of expertise without specific hypotheses to test. However, to use topic models effectively relies on choices about both text processing and model initialization. Without prior experience in machine learning and natural language processing, these choices may be challenging to navigate. I focus on two primary challenges in establishing datasets for effective topic models: pre-processing and privacy. In the first part, I share a number of experiments to discover the effects of standard text pre-processing steps on the learned topic models. My work shows common practices in text cleaning, including stemming, stopword removal, and text de-duplication may be less necessary than conventionally assumed. In the second part, I discuss a workflow to apply differential privacy through randomization for Poisson factorization models, a broad class of distributional models of count data including LDA. My work includes multiple methods for efficient inference of private Poisson factorization models on large datasets, including approximations to an MCMC algorithm and a new variational inference (VI) algorithm. I also discuss approaches to introduce of randomness to privatize unigram frequencies to better preserve the sparse, correlated structure of the true data.
Degree
thesis:*- Name thesis:degree_name
- Ph.D., Computer Science
- Level thesis:degree_level
- Doctor of Philosophy
- Discipline thesis:degree_discipline
- Computer Science
- Grantor
- Cornell University
- Year dc:date.issued
- 2019
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- SCHOFIELD, ALEXANDRA K
- Chair dc:contributor.chair
-
- Mimno, David
- Committee members dc:contributor.committeemember
-
- Danescu-Niculescu-Mizil, Cristian
- Birman, Kenneth Paul
Subjects
dc:subject × 7Rights
dc:rights- Statement dc:rights
-
- Attribution-ShareAlike 2.0 Generic
- Licence dc:rights.uri
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Dc Identifier Bibid
- 11050287
- Dc Identifier Other
-
ProQuest Submission ID: 11395
ProQuest Publication ID: 13881306