Back to search

Georgia Institute of Technology

Stochastic m-estimators: controlling accuracy-cost tradeoffs in machine learning

Abstract

dc:description.abstract

m-Estimation represents a broad class of estimators, including least-squares and maximum likelihood, and is a widely used tool for statistical inference. Its successful application however, often requires negotiating physical resources for desired levels of accuracy. These limiting factors, which we abstractly refer as costs, may be computational, such as time-limited cluster access for parameter learning, or they may be financial, such as purchasing human-labeled training data under a fixed budget. This thesis explores these accuracy- cost tradeoffs by proposing a family of estimators that maximizes a stochastic variation of the traditional m-estimator. Such "stochastic m-estimators" (SMEs) are constructed by stitching together different m-estimators, at random. Each such instantiation resolves the accuracy-cost tradeoff differently, and taken together they span a continuous spectrum of accuracy-cost tradeoff resolutions. We prove the consistency of the estimators and provide formulas for their asymptotic variance and statistical robustness. We also assess their cost for two concerns typical to machine learning: computational complexity and labeling expense. For the sake of concreteness, we discuss experimental results in the context of a variety of discriminative and generative Markov random fields, including Boltzmann machines, conditional random fields, model mixtures, etc. The theoretical and experimental studies demonstrate the effectiveness of the estimators when computational resources are insufficient or when obtaining additional labeled samples is necessary. We also demonstrate that in some cases the stochastic m-estimator is associated with robustness thereby increasing its statistical accuracy and representing a win-win.

Degree

thesis:*
Department dc:contributor.department
Computing
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2011

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Dillon, Joshua V.
Advisor dc:contributor.advisor
  • Gray, Alexander G.
Committee members dc:contributor.committeemember
  • Zha, Hongyuan
  • Collins-Thompson, Kevyn
  • Lebanon, Guy
  • Borodovsky, Mark

Subjects

dc:subject × 8

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/1853/42913

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Dillon, Joshua V.. Stochastic m-estimators: controlling accuracy-cost tradeoffs in machine learning. Georgia Institute of Technology, 2011. http://hdl.handle.net/1853/42913