Back to search

McMaster University

Online EM for Exponential Family Distributions

Abstract

dc:description.abstract

The expectation-maximization (EM) algorithm is a widely utilized method for fitting probabilistic models with missing data or latent variables. However, applying the EM algorithm to high-dimensional datasets requires storing the entire dataset in computer memory, which is necessary at each iteration. Fortunately, online learning algorithms are available that overcome this issue by processing data sequentially and updating model parameters at each step. In this thesis, we introduce an approach for online training of mixture models of exponential family distributions through modifications to the EM algorithm. The first part of our contribution is the learning of Gaussian mixture models (GMMs) using alternating mirror descent updates. This method employs custom Bregman divergences for parameter updates and allows for the processing of one unit of data at each iteration. Our second contribution is an improved version of the EM algorithm for exponential family distributions. In this case, we replace the M-step updates with mirror descent updates and further enhance them by introducing a new Bregman divergence for the updates. Finally, we apply the improved EM algorithm to online principal component analysis (PCA) for exponential families, where the number of principal components is adaptively learned by pruning factors with low correlations to the observations. We evaluate the performance of our proposed approaches by comparing them with both simulation and real datasets.

Degree

thesis:*
Department dc:contributor.department
Computational Engineering and Science
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Rahimipour, Shiva
Advisor dc:contributor.advisor
  • McNicholas, Paul D.

Subjects

dc:subject × 5

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
McMaster University
Base URL
macsphere.mcmaster.ca/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Rahimipour, Shiva. Online EM for Exponential Family Distributions. 2024. http://hdl.handle.net/11375/30425